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FOREWORD 


The  purpose  of  this  document  is  to  communicate  the  annual  progress  for  each  individual 
research  project  in  the  Research  and  Advanced  Concepts  Office  (RACO)  basic  research 
program  at  the  United  States  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI).  The  summaries  contained  herein  are  written  by  the  scientists  who  are 
performing  the  work  and  provide  a  snapshot  of  their  continuing  efforts.  In  addition, 
RACO  conducts  a  more  detailed  in-progress  review  of  each  project  each  year.  If 
successful,  the  projects  within  RACO’s  basic  research  program  will  lay  the  foundations 
for  many  of  ARI’s  future  applied  behavioral  research  efforts.  These  summaries  serve  as 
guideposts  for  ensuring  that  our  basic  research  results  transition  to  ARI’s  applied  research 
programs. 
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Zita  M.  Simutis 
Acting  Director 
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THE  BASIC  RESEARCH  PROGRAM: 

Identifying  New  and  Promising  Technologies 

The  Research  and  Advanced  Concepts  Office  (RACO)  manages  ARI’s  basic  research  program 
that  focuses  on  combinations  of  the  personnel,  leadership,  and  training  requirements  of  the 
future.  The  basic  research  program  is  a  critical  link  between  the  world  of  behavioral  science  and 
the  military  community.  Searching  out  and  advancing  the  state-of-the-art  methods,  theories  and 
findings  in  behavioral  science,  encouraging  projects  most  likely  to  contribute  generalizable 
scientific  principles  and  new  knowledge,  and  supporting  those  efforts  that  have  potential  military 
relevance  and  likelihood  of  leading  to  applied  behavioral  technology  are  RACO  s  key  research 
goals. 

Universities  conduct  most  of  the  basic  research  program.  RACO  maintains  close  contact  with 
ARI’s  applied  researchers  and  other  relevant  agencies  within  the  Army.  These  contacts  help  to 
define  issues  that  require  fundamental  research,  and  facilitate  the  transition  of  basic  research 
results  to  applied  programs  for  eventual  use  by  the  operational  Army. 

In  RACO’s  contract  programs,  a  Broad  Agency  Announcement  (BAA)  is  issued  each  year  to 
solicit  both  concept  papers  and  formal  proposals  relating  to  the  announced  program  areas.  In  a 
given  year,  the  BAA  highlights  the  research  objectives  of  special  interest,  and  provides  an  open 
call  for  proposals. 

On  the  following  pages,  the  reader  will  find  summaries  of  current,  on-going  RACO  contracts 
ranging  over  the  period  1998  -  2002.  There  are  three  current  RACO  program  objectives,  each  of 
which  will  be  discussed  in  detail  below.  They  are  as  follows: 

1.  Provide  fundamental  knowledge  to  improve  training  in  complex,  digital  environments, 

2.  Provide  fundamental  knowledge  to  improve  leader  assessment  and  accelerate  leader 
development, 

3.  Provide  fundamental  knowledge  for  identifying  and  measuring  the  attributes  and  skills 
that  are  critical  to  soldier  recruiting,  performance,  and  retention  in  the  transforming 
Army. 

Twining  for  Speed  in  Knowledge 

Basic  research  in  this  area  focuses  on  developing  concepts  and  methods  for  training  complex 
tasks  and  for  sustaining  complex  task  performance.  Assessing  the  cognitive  impact  of  Objective 
Force  technology  requirements  arising  from  digital  systems,  semi-automated  systems,  and 
robotic  systems,  on  training  requirements  is  also  part  of  this  process.  A  new  effort  in  this  area 
seeks  to  understand  the  impact  of  the  voluminous  multi-modal  data  on  performance  and  how 
individuals  and  teams  might  be  trained  to  integrate  and  use  such  rapidly  presented  data.  We  seek 
to  identify  unique  training  principles  and  methods  for  improving  interpersonal  skills  and  team 
adaptability  and  performance.  The  expected  outcome  is  applied  research  testing  the  principles 
and  methods  produced  in  this  work  package  in  Army  training  environments.  The  models  and 
theories  produced  should  be  useful  in  accounting  for  individual  differences  in  training  and 
facilitating  practical,  individualized,  adaptive  training  methods.  These  advances  will  translate 
into  improved  training  methods  in  a  wide  range  of  tasks. 
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Assessing  and  Improving  Leader  Skills 

Commensurate  with  the  Objective  Force  requirements  for  rapidly  developing  adaptable,  flexible 
leaders,  the  basic  research  program  in  leader  development  is  directed  toward  providing  concepts 
and  methods  for  accelerating  leader  development  and  understanding  and  developing  leader 
adaptability  and  flexibility  in  a  manner  that  can  be  tested  in  the  applied  environment.  One  of  our 
major  efforts  in  this  area  is  using  Sternberg’s  theory  of  practical  intelligence  to  develop  new 
techniques  for  acquiring  experience-based,  tacit  knowledge  as  a  means  for  rapidly  developing 
effective  military  leaders.  We  are  also  committed  to  discovering  and  testing  the  basic  cognitive 
principles  that  underlie  effective  leader-team  performance.  Understanding  the  dynamics  of  small 
group  leadership  in  face-to-face  and  distributed  team  environments  is  a  key  aspect  current  basic 
research  program  in  leadership.  For  example,  one  of  our  research  projects  seeks  to  develop 
leadership  techniques  that  foster  interaction,  communication,  and  trust  in  electronic 
environments.  Another  examines  distant  leadership  behaviors  under  the  stress  of  performing  in  a 
metropolitan  hospital  shock  trauma  center. 

Personnel  Issues  for  the  New  Century, 

Identifying  and  measuring  the  aptitudes  and  skills  that  are  unique  to  the  human  performance 
requirements  of  the  Objective  Force  is  a  major  theme  of  this  basic  research  effort.  As  part  of  this 
process,  we  seek  to  devise  methods  that  assess  persistency  and  dependability,  describe  how  these 
attributes  develop,  and  measure  their  contribution  to  performance  and  job  tenure.  For  example, 
we  have  begun  a  new  effort  to  assess  individual  adaptability  and  flexibility  as  aptitudes  that  can 
be  measured  across  job  domains.  Exploring  the  sociological  and  psychological  factors  that  could 
influence  recruitment,  retention,  and  performance  is  part  of  this  work  package  as  well.  Several 
sociological  research  efforts  seek  to  understand  how  various  social  structures,  such  as  the  family, 
and  population  demographics  influence  Army  performance.  In  another  research  effort  we  are 
investigating  the  conditions  under  which  turnover  hinders  or  helps  team  performance.  Although 
turnover  can  be  harmful,  for  example,  when  a  team  loses  a  productive  member  and  must  expend 
time  and  energy  training  a  replacement;  turnover  can  also  be  helpful,  as  when  a  team  loses  an 
unproductive  member  and  gains  a  replacement  with  valuable  knowledge.  We  anticipate  that 
results  from  this  research  will  make  an  important  contribution  to  understanding  and  improving 
organizational  effectiveness. 

This  document  provides  a  listing  and  brief  synopsis  of  ongoing  and  recently  completed  research 
efforts.  Project  listings  are  organized  into  the  three  aforementioned  research  objectives.  It  is 
important  to  note,  however,  that  this  program  is  but  one  of  many  programs  for  which  RACO  has 
responsibility.  Other  programs  in  RACO  include; 

•  Small  Business  Innovative  Research  (SBIR)  Program, 

•  Small  Business  Technology  Transfer  (STTR)  Program, 

•  International  Behavioral  Science  and  Technology  Watch, 

•  Graduate  student  apprenticeship  program  -  Consortium  Research  Fellows  Program  -  with  the 
Consortium  of  Metropolitan  Washington  Universities, 

•  Outreach  efforts  to  Historically  Black  Colleges  and  Universities 
(HBCUs)  and  Minority  Institutions  (Mis),  and 

•  Research  support  in  behavioral  science  for  the  U.S.  Military  Academy. 

Additional  information  about  reports  from  these  research  efforts  is  available  upon  request. 
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RACO  RESEARCH  OBJECTIVE  #1: 


Provide  fundamental  knowledge  to  improve  training  in  complex, 
digital  environments. 


Research  under  this  objective  develops  concepts  and  methods  for 
training  complex  tasks  and  sustaining  complex  task  performance.  The 
focus  is  on  understanding  the  cognitive  impact  of  Objective  Force 
technology  on  training  requirements,  the  impact  of  voluminous  multi¬ 
modal  data  on  performance,  and  developing  methods  for  improving 
the  interpersonal  and  cognitive  skills  that  affect  team  performance. 
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Optimizing  the  Speed,  Durability,  and  Transferability  of  Training 

Contract  #:  DASW01-99-K-0002  Contract  Dates:  8/1/99  -  7/31/02 

Institution:  University  of  Colorado  Pis:  Alice  F.  Healy  &  Lyle  E.  Bourne,  Jr. 

Scientific  Objectives 

The  U.  S.  Army  spends  much  time  and  many  resources  in  training  its  personnel.  Training 
is  essential  because  recruits  cannot  be  expected  to  come  equipped  with  the  military  knowledge 
and  skills  they  will  need  in  the  field.  But  training  is  costly,  so  it  is  important  to  ensure  that  it  be 
accomplished  as  quickly  and  efficiently  as  possible.  However,  increasing  training  speed  should 
not  be  the  only,  or  even  the  most  important,  consideration.  If  soldiers  successfully  learn  how  to 
perform  a  task  during  training,  but  then  forget  how  to  perform  it  when  needed,  training  has 
clearly  been  inadequate.  Passing  a  test  at  the  end  of  training  does  not  guarantee  later  success  in 
the  field.  Training  needs  to  be  durable  as  well  as  efficient.  But  even  durable  training  cannot 
guarantee  that  learned  knowledge  and  skills  will  be  applied  successfully  to  situations  different 
from  those  encountered  during  training.  Training  circumstances  can  rarely  capture  the  full  set  of 
circumstances  under  which  tasks  are  encountered  in  the  field.  Therefore,  it  is  essential  that 
training  be  transferable  as  well  as  durable.  It  is  the  aim  of  our  research  program  to  develop 
principles  that  separately  optimize  the  three  major  aspects  of  training:  (1)  speed/efficiency,  (2) 
durability/long-term  retention,  and  (3)  transferability  or  generalizability  to  new  situations. 

Approach 

Although  many  of  our  studies  have  overlapping  goals,  for  present  paper  we  have  divided 
them  into  three  major  groups.  The  first  group  is  principally  concerned  with  managing  factual 
overload,  rapidly  presented  information,  stress,  frustration,  and  fatigue,  with  an  emphasis  on 
tasks  involving  perceptual  and  motor  processing.  The  second  group  addresses  the 
comprehension  of  verbal  and  spatial  information.  The  last  group  primarily  considers  optimizing 
the  balance  of  the  three  major  aspects  of  training. 

Studies  on  managing  factual  overload,  rapidly  presented  information,  stress,  frustration, 
and  fatigue  focus  on  the  following  four  issues:  (1)  handling  factual  overload,  (2)  dealing  with 
rapidly  presented  information,  (3)  initiating  and  executing  responses  under  stress  and  frustration, 
and  (4)  initiating  and  executing  response  components  under  fatigue  produced  by  prolonged 
work. 

Studies  on  comprehension  of  verbal  and  spatial  information  focus  on  the  following  four 
issues:  (1)  how  the  mental  representation  of  space  affects  the  comprehension  of  navigation 
directions  within  that  space,  (2)  the  retention  and  transfer  of  the  skills  involved  in 
comprehending  navigation  directions,  (3)  the  relationship  between  the  verbal  and  spatial 
representations  of  navigation  directions,  and  (4)  the  effects  of  overload  on  the  comprehension  of 
navigation  instructions. 

Studies  on  balancing  the  three  major  aspects  of  training  focus  on  the  following  four 
issues:  (1)  the  sources  of  specificity  during  training  and  transfer,  (2)  variations  in  contextual 
interference  and  task  difficulty  as  a  means  to  promote  durability  and  transferability  of  training, 

(3)  types  of  strategies  and  strategy  transitions  during  training,  retention,  and  transfer,  and  (4) 
seeding  the  knowledge  base  with  a  minimal  set  of  facts. 
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Progress 


Managing  factual  overload,  rapidly  presented  information,  stress,  frustration,  and  fatigue 

We  studied  response  execution  in  a  motor  coordination  task  involving  various  forms  of 
hand-eye  incompatibility.  We  found  that  practice  with  a  given  form  of  incompatibility  led  to 
large  improvements  during  training  and  high  durability  across  a  1-week  delay,  but  minimal 
transfer  from  one  type  of  incompatibility  to  another.  Even  when  subjects  practiced  with  a 
condition  that  combined  up-down  and  right-left  incompatibility,  there  was  no  transfer  to 
performance  with  a  condition  that  involved  only  one  of  the  types  of  incompatibility.  Also, 
training  with  the  more  difficult  right-left  incompatibility  did  not  aid  in  performance  with  the 
simpler  up-down  incompatibility.  The  current  implication  for  military  training  is  that  when  eye- 
hand  coordination  is  involved  in  a  skill,  for  practice  to  be  effective,  specific  forms  of  hand-eye 
incompatibility  must  be  trained  in  the  particular  way  that  will  be  encountered  subsequently  in  the 
field.  Our  future  research  on  this  topic  will  be  aimed  at  developing  methods  to  overcome  this 
specificity,  to  enable  trainees  to  adapt  to  new  conditions  not  encountered  during  practice. 

Comprehension  of  verbal  and  spatial  information 

We  have  developed  a  laboratory  analog  of  communication  between  air  traffic  controllers 
and  pilots.  In  this  task,  participants  are  given  instructions  like  those  given  by  air  traffic 
controllers.  Sometimes  they  were  required  to  repeat  the  instructions  aloud,  as  pilots  are  expected 
to  do,  and  then  follow  the  directions  to  navigate  in  the  space  displayed  on  the  computer.  Using 
this  analog,  we  completed  an  experiment  in  which  we  studied  the  effects  of  presentation  format 
and  readback  on  the  comprehension  and  execution  of  navigation  instructions.  As  in  previous 
studies,  we  found  a  large  drop  in  performance  between  messages  with  three  and  messages  with 
four  commands,  bolstering  our  earlier  recommendation  to  limit  navigational  instructions  to  three 
commands  if  possible.  In  addition,  as  in  a  previous  study,  we  found  that  requiring  subjects  to 
readback  navigation  instructions  before  executing  them  led  to  a  large  decrement  in  their  ability  to 
follow  the  instructions.  However,  we  also  found  that  reducing  the  readback  requirement  by 
telling  subjects  to  readback  only  the  crucial  words  greatly  lessened  this  decrement.  Because  the 
advantage  for  reduced  readback  may  be  attributed  in  part  to  a  change  in  the  visual  scanning 
strategy,  we  only  tentatively  make  the  recommendation  for  pilots  and  others  receiving  navigation 
instructions  auditorily  to  reduce  the  length  of  their  readback.  Such  reduction  in  readback  will 
still  permit  verification  but  will  minimize  the  cost  of  output  interference.  Although  there  were 
no  overall  differences  between  simultaneous  and  sequential  presentation  of  the  commands,  with 
simultaneous  presentation,  there  was  a  greater  decline  in  performance  as  command  position 
increased,  suggesting  that  subjects  did  not  optimally  allocate  their  attention.  Therefore,  we 
recommend  that  individuals  who  must  follow  navigation  instructions  be  given  additional  training 
both  in  telescoping  their  readback  and  in  efficiently  allocating  their  attention  to  each  command 
of  the  instructions. 

Optimizing  the  balance  of  trainine  aspects 

We  studied  variations  in  contextual  interference  and  task  difficulty  as  a  means  to  promote 
durability  and  transferability  of  training.  We  found  that  increasing  task  difficulty  by  different 
manipulations  during  study  slowed  down  learning  but  seemed  to  aid  transfer  to  a  new  task  as 
well  as  long-term  retention  of  the  original  task.  However,  the  long-term  benefit  was  limited  to 
the  specific  trained  items  and  did  not  extend  to  other  items,  even  those  from  the  same  category  as 
the  trained  ones.  The  implication  for  training  is  that  enhancing  the  difficulty  of  training 
exercises  may  lead  to  improved  retention  and  transfer  to  new  tasks  but  not  to  new  items. 


5 


In  concept-learning  tasks,  participants  sometimes  adopt  an  initial  strategy  of  learning 
arbitrary  associations  between  items  and  responses,  but  later  make  a  transition  to  rule-based 
performance  once  the  concept  is  discovered.  In  mental  arithmetic  tasks,  participants  often  show 
a  transition  from  initial  rule-based  performance  to  memory-based  performance  after  extensive 
practice.  We  have  demonstrated  both  types  of  transitions  within  the  same  binary  classification 
task.  We  found  that  both  rule  difficulty  and  rule  salience  determine  the  strategy  that  subjects 
eventually  adopt  with  training  on  a  given  task.  In  a  new  experiment,  where  the  same  simple  rule 
was  used  in  all  tasks,  the  strategy  adopted  was  rule-based  or  instance-based,  depending  on  the 
salience  of  the  rule.  These  findings  imply  that,  to  optimize  performance,  trainers  need  to  be 
aware  of  both  the  salience  and  the  difficulty  of  the  operative  rule  for  responding  in  any  given 
task.  No  matter  how  simple  it  is,  when  a  rule  is  not  obvious  learners  may  adopt  a  strategy  in 
which  they  memorize  specific  instances  rather  than  use  the  underlying  rule. 

Contributions  to  Basic  Science 

The  many  research  directions  we  are  pursuing  address  a  diverse  set  of  issues  but  have 
three  primary  goals.  First,  we  have  initiated  an  intensive  examination  of  extraneous  variables 
(including  factual  overload,  rapid  presentation  of  information,  stress,  frustration,  and  fatigue) 
that  might  adversely  affect  training.  Understanding  the  impact  of  these  variables  will  enable  us 
to  develop  procedures  to  counteract  their  deleterious  effects.  Second,  we  have  commenced  a 
series  of  studies  that  could  lead  to  the  development  of  training  routines  for  optimizing  the 
comprehension  of  both  verbal  and  spatial  information.  Third,  we  have  started  research 
concerned  with  optimizing  the  relationship  between  all  three  major  aspects  of  training— its  speed, 
durability,  and  transferability.  These  studies  share  a  common  analytical  and  experimental 
methodological  approach  and  the  common  theoretical  goal  of  understanding  the  psychological 
principles  underlying  the  acquisition,  retention,  and  generalization  of  knowledge  and  skills. 

Potential  Applications 

These  studies  also  share  the  common  applied  goal  to  improve  the  training  of  military 
personnel  such  that  knowledge  and  skills  will  be  acquired  more  quickly,  yet  still  be  accessible 
across  long  delay  intervals  without  practice  as  well  as  adaptable  to  new  situations  outside  the 
training  environment.  The  balance  of  the  three  aspects  of  training  (speed,  durability,  and 
transferability)  is  not  necessarily  fixed  across  tasks  or  even  within  a  given  task.  Instead,  the 
balance  may  depend  on  a  variety  of  external  factors  that  can  change  from  time  to  time,  such  as 
stress,  frustration,  fatigue,  speed  of  information  presentation,  and  information  load.  Variations  in 
any  of  these  external  factors  can  affect  the  interaction  of  the  speed,  durability,  and  transferability 
of  training.  Hence,  our  research  also  examines  these  three  aspects  of  training  under  various 
conditions  of  stress,  frustration,  fatigue,  speed  of  information  presentation,  and  information  load. 
The  information  gathered  from  this  research  will  help  in  the  design  of  training  programs  that 
allow  the  trainee  to  contend  with  external  conditions  in  the  field  that  are  often  unpredictable  and 
could  adversely  influence  performance  if  not  properly  managed. 
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Understanding  aspects  of  individual  and  collaborative  skill  acquisition 
in  face-to-face  and  distance  training  situations 

Contract#:  DASW01-01-K0003  Contract  Date:  9/1/01-8/31/04 

Institution:  New  Mexico  State  University  PI:  Adrienne  Y.  Lee 

Co-PIs:  Douglas  Gillan  &  Nancy  Cooke 

Scientific  Objectives 

In  recent  years,  the  advancement  of  military  technology  has  resulted  in  a  need  to  acquire 
and  maintain  highly  complex  skills  to  effectively  use  that  technology  (Barry  &  Runyan,  1995). 
Consequently,  the  demand  for  ongoing  innovations,  including  distance  training,  has  increased. 
Distance  training  has  been  shown  to  be  extremely  cost  effective  and  has  produced  similar 
learning  outcomes.  Although  extensive  research  on  distance  learning  exists,  few  studies  have 
focused  on  group  (collaborative)  distance  training  delivered  by  Intemet/web-based  technologies. 

A  parallel  development  has  occurred  in  the  area  of  team  training.  Even  though  group 
(collaborative)  learning  has  been  the  focus  of  many  studies  (Slavin,  1996)  and  progress  has  been 
made  on  knowledge  measurement  at  the  individual  level,  the  measurement  of  team  knowledge 
and  team  cognition  is  still  in  its  infancy  (Cooke,  Salas,  Cannon-Bowers  &  Stout,  2000).  Thus, 
research  on  how  to  improve  team  distance  learning  could  contribute  significantly  to  the  field. 

The  questions  being  addressed  in  our  research  include:  (1)  Do  co-located  teams  train 
more  quickly  than  distributed  teams?  (2)  When  tested  in  the  same  situation  (e.g.,  co-located 
training  and  testing  and  distributed  training  and  testing),  will  co-located  teams  perform  the 
transfer  mission  more  accurately  and  learn  more  quickly  than  distributed  teams?  (3)  Will  the 
results  be  different  when  teams  are  asked  to  learn  a  new  transfer  mission  in  a  different  situation 
(e.g.,  co-located  learning  to  distributed  testing  and  distributed  learning  to  co-located  testing)? 
Additionally,  in  a  follow-up  experiment,  we  will  be  adding  training  based  upon  observations  and 
data  collected  in  experiment  1.  Thus,  an  additional  question  includes  whether  this  added  training 
improves  performance  for  teams  learning  in  a  distributed  manner. 

Approach 

This  research  examines  the  acquisition  of  complex  skills  by  teams  either  training  face-to- 
face  (co-located)  or  remotely  (distance/distributed  training).  In  face-to-face  training,  team 
members  are  co-located  and  interactions  are  encouraged  through  the  physical  arrangement  of  the 
computers  (training  systems).  In  distance  training,  team  members  located  at  different  locations 
from  the  instructor  and  each  other.  Two  experiments  are  being  conducted.  In  the  first 
experiment,  we  are  testing  teams  in  the  same  or  different  training  situation  to  assess  the  degree  of 
team-relevant  skills  transfer.  The  second  experiment  will  examine  task-related  transfer  when 
varying  the  test  task  relative  to  the  training  task.  In  addition,  if  distance  training  is  not  better 
than  face-to-face  training,  then  the  second  experiment  will  draw  from  the  first  experiment  to 
improve  distance  learning. 


Progress 

We  have  designed  the  materials  and  experimental  protocol  for  the  first  experiment.  We 
have  created  a  listing  of  specified  changes  to  the  equipment/computers  and  these  changes  have 
been  delivered  to  Sandia  Research  for  implementation.  We  have  determined  the  details  of  the 
experimental  protocol  and  what  test  instruments  need  to  be  delivered  by  the  experimenters. 
These  changes  will  be  made  to  the  experimenter’s  manual.  Further,  student  experimenters  have 
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been  trained  and  are  participating  in  the  current  experiment  as  practice  for  running  subjects  on 
the  first  experiment.  We  anticipate  that  we  will  start  to  run  subjects  for  this  experiment  by  the 
end  of  March. 


Contributions  to  Basic  Science 

This  research  lies  at  the  intersection  of  two  current  issues  in  training  -  team  cognition  and 
distance  learning.  Although  extensive  research  has  been  performed  on  distance  and  group 
(collaborative)  learning,  few  studies  have  focused  on  collaborative  distance  learning  and  only 
recently  have  studies  focused  on  team  cognition.  Even  fewer  studies  have  been  performed  on 
team  distance  learning  (where  team  members  are  located  at  a  distance  from  each  other  and  the 
instructor).  Thus,  this  research  has  implications  for  the  advancement  of  scientific  knowledge 
concerning  team  learning,  distance  learning  and  team  distance  learning.  It  will  also  provide 
specific  advice  about  enhancing  these  three  types  of  learning. 

Potential  Applications 

A  significant  amount  of  the  success  in  military  performance  is  due  to  team  performance, 
rather  than  just  individual  performance.  Therefore,  providing  effective  team  training  is  critical. 
Because  Internet  technology  has  improved  dramatically  over  the  last  decade,  distance  learning 
that  includes  high  quality  audio  and  visual  transmission  is  possible.  Web-based  instruction 
provides  an  opportunity  to  cut  costs  and  deliver  training  to  individuals  at  remote  locations  before 
they  must  either  learn  together  as  a  team  or  perform  some  team  task.  Because  remote  training 
can  be  cost  effective  and  recent  advances  in  Intemet/web-based  technology  allow  for 
interactivity,  research  on  how  to  improve  team  distance  learning  will  be  beneficial  for  the  Army. 
Further,  in  today’s  military,  teams  of  individuals  who  may  never  have  met  each  other,  who  are 
not  necessarily  sitting  together  in  the  same  briefing  room  side-by-side  on  the  same  battlefield, 
and  who  may  only  communicate  and  share  information  via  communication  and  computer 
technologies  perform  many  military  tasks.  Indeed,  the  entire  nature  of  warfare  has  taken  on  a 
“network  centric”  characteristic  (Wilson,  2000)  in  which  the  battlefield  is  dispersed  not  only 
over  terrain  but  also  over  the  Internet.  Thus,  these  studies  will  provide  a  better  understanding  of 
team  learning  (cognition  and  performance)  that  can  be  used  to  improve  both  face-to-face  and 
distance  team  training. 
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The  Effect  of  Team  Training  on  Team  Cohesion 

Contract  #:  DASW01-00-K-0013  Contract  Dates:  8/14/01-10/13/02 

Institution:  Old  Dominion  University  PI  Name:  Robert  M.  McIntyre 

Scientific  Objections 

The  objectives  of  this  research  are  (a)  to  empirically  examine  the  effects  of  team  process 
training  on  team  cohesiveness,  (b)  to  empirically  investigate  the  relationship  between  team 
cohesiveness  and  team  performance.  The  team  process  training  was  based  on  a  model  of  team 
processes  developed  by  Dickinson  and  McIntyre  (1997). 

Approach 

We  have  approached  this  research  by  applying  two  different  paradigms.  The  first 
involves  the  use  of  a  standard  pre-post  control  group  study  with  randomization  of  teams  to  either 
a  placebo  or  a  team  training  condition.  The  dependent  variables  investigated  in  this  first 
approach  were  the  components  of  cohesion  as  measured  by  the  System  for  the  Multiple  Level 
Observation  of  Group  (SYMLOG)  (Bales  &  Cohen,  1980).  A  four-hour  training  program  based 
on  the  team  process  model  was  conducted  with  the  experimental  group.  A  manipulation  check 
on  the  team  training  was  earned  out  by  applying  a  measure  of  the  seven  team  processes 
developed  by  Rosenstein  (1994).  Results  indicated  that  cohesion  was  greater  immediately  after 
training  as  well  as  one  month  after  the  training  program  had  ended  when  compared  to  the  control 
group.  This  suggested  that  team  process  training  can  contribute  to  greater  levels  of  team 
cohesion. 

The  second  paradigm  is  based  on  a  true  time  series  design.  It  examines  (a)  team  cohesion 
as  a  function  of  time,  (b)  the  effects  of  team  training  on  team  cohesion,  (c)  the  decay  of  team 
cohesion  over  time,  and  (d)  the  time-dependent  relationship  between  team  cohesion  and  team 
performance.  "Tme  dependency"  refers  to  the  tendency  for  time  series  data  to  be  autocorrelated. 
The  use  of  the  time  series  design  provides  the  basis  for  answering  multiple  questions 
simultaneously  including  questions  about  team  cohesiveness,  team  training,  and  team 
performance  in  ways  that  have  not  been  used  in  the  past.  In  addition,  we  are  examining  a  set  of 
variables  that  may  impact  the  relationships  between  team  training,  team  performance,  and  team 
cohesiveness.  These  variables  include  affect  of  team  members  (positive  and  negative),  team 
members’  perceptions  of  task  load,  team  members’  perceptions  of  “life  load,  diversity  of  team 
members  vis  a  vis  demographics,  and  diversity  of  team  members  vis  a  vis  academic 
achievement. 

In  the  time  series  experiment,  1 1  teams  of  students  (each  consisting  of  five  or  six 
students)  participated  over  a  period  of  15  weeks  in  a  course  designed  around  the  team  concept. 
That  is,  during  the  course  students  learned  by  self-study  and  through  team  exercises.  Teams  met 
three  times  per  week.  Students  worked  on  a  multi-part  class  assignment  for  50  minutes.  This 
assignment  was  scored  and  is  treated  as  the  team  performance  measurement.  In  addition,  at  the 
end  of  each  session,  team  members  complete  a  series  of  questionnaires  designed  to  assess 
cohesiveness,  affect,  and  pepreeived  work  load. 

Team  training  was  provided  during  the  eighth  week  of  the  course.  This  training  was 
based  on  the  Dickinson-Mclntyre  team  process  model.  The  format  of  the  training  included 
lecture,  modeling,  practice,  and  feedback.  It  was  hypothesized  that  team  performance  and  team 
cohesion  would  improve  as  a  function  of  team  training.  To  assess  the  effect,  a  separate  time 
series  analysis  was  carried  out  for  each  team. 
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Progress 

The  first  experiment  (involving  the  use  of  the  pre-post  training  design)  was  completed  in 
2001.  We  presented  the  results  of  this  study  at  the  17  annual  conference  of  the  Society  for 
Industrial  and  Organizational  Psychology. 

The  second  three-phase  study  was  carried  out  within  the  context  of  three  Masters  theses. 
The  breakdown  into  three  phases  was  implemented  because  of  the  complexity  and  difficulty  of 
the  analyses.  Two  of  the  three  are  complete.  In  the  first,  the  results  indicated  that  team  training 
positively  affected  cohesiveness.  However,  training  did  not  affect  team  performance.  In  the 
second  study,  a  variable  set  of  results  were  found.  For  certain  teams,  affect-pertinent  variables 
affected  cohesiveness.  However,  no  performance  predictors  were  found. 

These  somewhat  puzzling  results  led  us  to  examine  the  nature  of  our  performance 
measure  and  the  observations  we  had  regarding  how  the  student  learning  teams  approached  the 
team  task.  The  learning  teams  did  not  like  the  course  structure.  Student  teams  broke  the  team 
tasks  into  disjunctive  parts,  thereby  making  the  task  much  less  team-dependent  than  planned.  In 
addition,  we  noted  that  the  performance  measure  may  suffer  from  validity  problems  in  that 
course  material  changed  each  week  which  may  have  resulted  in  exercises  that  were  not 
uniformly  difficult.  To  address  these  issues  and  to  explore  other  more  complex  models,  the  third 
phase  of  the  research  is  being  carried  out.  It  will  be  completed  as  a  part  of  Hope  Hanner’s 
Masters  thesis  before  the  end  of  August.  The  final  report  will  be  written  and  results  presented  to 
ARI  in  October. 

In  addition  to  the  work  described  above,  the  research  team  at  ODU  has  carried  out  several 
additional  endeavors.  First,  the  content  validity  of  the  training  program  was  established  to 
ensure  it  was  representative  of  the  material  that  should  be  presented  in  a  team  training  program. 
Second,  we  designed  a  prototype  of  an  interactive  computer-assisted  team  training  device  as  a 
proof  of  concept.  Third,  we  started  to  revise  the  team  training  program  for  the  next  phase  of  our 
research.  Fourth,  the  director  of  the  ROTC  students  at  ODU  was  contacted  in  order  to  elicit  his 
support  in  future  research.  We  plan  to  continue  our  research  program  and  to  include  a  stronger 
emphasis  on  leadership  in  team  training.  Thus,  although  the  Dickinson-Mclntyre  process  model 
includes  leadership  as  a  critical  component,  it  will  receive  much  greater  emphasis  in  the  future. 
Fifth,  we  are  searching  for  team  tasks  on  which  to  study  team  performance.  Within  the  context 
of  the  current  contract,  it  is  impossible  to  purchase  team  tasks.  However,  there  are  computer- 
based  tasks  (such  as  the  microworld  used  for  training  FAA  personnel)  that  would  be  both 
realistic  and  practical  team  training  research  tasks.  We  hope  to  access  such  tasks  in  the  future. 

Contributions  To  Basic  Science 

The  first  study  implied  that  model-driven  team  training  affected  team  performance  and 
the  effect  persisted  over  time.  Therefore,  the  main  contribution  of  this  study  is  the  finding  that 
when  one  uses  a  coherent  and  data-based  conception  of  teamwork,  one  can  expect  to  find 
increased  cohesion. 

The  second  study  is  much  more  complex.  This  three-phase  research  design  is  intended  to 
address  the  following:  (a)  how  a  team  becomes  cohesive,  (b)  whether  cohesion  covaries  over 
time  with  performance  (taking  into  account  autocorrelation  of  data),  (c)  the  nature  of  the  effect  of 
team  training  on  team  performance  and  team  cohesiveness  over  time,  (d)  the  effect  of  workload 
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on  cohesiveness  and  performance,  and  (e)  the  effect  of  affective  state  on  team  training,  cohesion, 
and  performance. 

The  examination  of  team  performance  and  team  processes  over  time  is  an  important  and 
unexplored  area.  Ultimately,  it  will  have  a  major  bearing  on  understanding  team  leadership.  At 
present,  the  contributions  are  still  unclear  from  this  complex  study.  For  example,  although 
certain  relationships  were  found  in  phase  one  and  two  of  this  study,  these  were  not  consistent 
across  teams.  It  is  unclear  how  to  deal  with  the  inconsistency.  Classical  science  may  suggest 
that  if  a  phenomenon  is  inconsistent,  it  is  not  reliable,  and  therefore  is  unreal.  However,  we  are 
not  quick  to  draw  such  a  conclusion.  Perhaps  what  basic  science  will  learn  from  our  research  is 
that  certain  relationships  exist  for  some  teams  and  not  for  others.  This  points  to  other  basic 
questions  that  beg  to  be  answered.  The  third-phase  study  will  address  some  of  these  questions. 

Potential  Applications 

Our  research  provides  a  framework  for  developing  effective  team  training  systems.  In 
fact,  our  vision  is  that  from  the  earliest  stages  of  socialization  a  soldier  may  begin  to  learn  to  be  a 
team  member  through  the  perspective  of  the  teamwork  process  model.  Admittedly,  this  vision 
goes  beyond  the  current  findings.  However,  the  results  thus  far  do  suggest  that  learning  about 
teamwork  can  provide  the  basis  for  making  certain  behavioral  changes  that  will  lead  to  increased 
cohesiveness.  We  believe  that  if  a  systematic  training  program  were  developed  to  provide  more 
modeling,  practice,  and  feedback,  it  would  prove  to  be  an  important  tool  for  increasing 
cohesiveness.  Further,  although  our  data  are  still  not  supportive  of  the  following  statement  (due 
to  unforeseen  measurement  problems  in  the  second  study),  we  continue  to  hypothesize  that  with 
the  proper  task,  the  appropriate  population,  and  the  appropriate  team  performance  measure,  team 
process  training  will  lead  to  improved  performance  and  cohesion.  In  effect,  the  studies  that  we 
carried  out,  with  a  relatively  small  amount  of  resources,  serve  as  a  proof  of  concept  that  team 
process  training  can  be  implemented  and  can  lead  to  positive  outcomes  for  the  field. 
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Working  Memory  Influences 
on  Long-Term  Memory  and  Comprehension 

Contract  #:  DASW01-99-K-0001  Contract  Dates:  06/15/99-06/14/02 

Institution:  University  of  Notre  Dame  PI:  Gabriel  A.  Radvansky 

Scientific  Objectives 

An  important  need  of  a  large  organization,  such  as  the  Army,  is  that  complex  sets  of 
information  about  events  in  the  world  be  readily  understood  by  the  people  entrusted  with  that 
information.  This  would  include  the  ability  to  understand  descriptions  of  events  occurring  in 
some  other  part  of  the  world  and  the  need  to  understand  and  accurately  remember  instructions 
about  how  to  perform  under  various  circumstances.  To  understand  information  well,  a  person 
must  be  able  to  successfully  comprehend  the  nature  of  the  events  that  are  being  referred  to  by  the 
descriptions  that  have  been  provided.  The  simple  retention  of  a  collection  of  individual  pieces  of 
information  is  insufficient.  A  person  must  be  able  to  properly  relate  that  information  to  external 
circumstances.  This  project  has  two  primary  goals.  First,  to  assess  how  an  individual’s  cognitive 
abilities  can  be  used  to  predict  future  performance  on  comprehension  and  long-term  memory 
tasks.  Second,  to  develop  methods  of  training  that  are  tailor-made  to  an  individual’s  weaknesses. 
Specifically,  this  project  focuses  on  how  working  memory  (that  portion  of  cognition  where 
information  is  actively  manipulated)  influences  higher-level  comprehension  and  the  long-term 
memories  that  result  from  this  comprehension.  These  long-term  memories  are  often  called 
mental  models  because  they  are  mental  simulations  of  sets  of  circumstances  that  could  exist  in 
the  world.  This  project  assesses  the  degree  to  which  successful  processing  of  mental  models  is 
predicted  by  traditional  measures  of  cognitive  ability  or  whether  this  processing  can  be  better 
predicted  by  measures  aimed  more  directly  at  how  these  representations  are  created  and 
retrieved.  Several  studies  are  being  conducted  to  examine  how  various  ability  tests  can  be  used 
to  predict  future  performance  in  processing  event  and  situation  information.  This  is  important 
because  our  knowledge  about  events  in  the  world  relies  more  on  the  mental  models  we  create 
than  on  lower  level  representations,  such  as  propositional  codes.  Mental  models  are  a  means  of 
capturing  and  coordinating  complex  sets  of  information  about  circumstances  and  events. 

Approach 

This  project  involves  a  series  of  experimental  and  correlational  studies  that  examine 
individual  differences  in  complex  information  understanding  and  one’s  subsequent  memory  for 
that  information.  For  the  project,  we  utilize  various  individual  difference  measures  of  cognitive 
ability  as  well  as  evaluate  performance  on  various  language  comprehension  and  memory 
retrieval  tasks  that  assess  complex  information  processing  about  events  and  episodes.  The 
cognitive  ability  measures  are  designed  to  quantify  a  person’s  basic  cognitive  skills.  In 
comparison,  the  performance  measures  are  designed  to  assess  the  actual  processing  of  complex 
sets  of  information  both  during  the  actual  process  of  understanding,  as  well  as  later,  when 
individuals  must  remember  needed  information.  These  performance  measures  cover  a  range  of 
mental  tasks  including  language  comprehension,  long-term  memory,  and  logical  reasoning.  The 
data  for  this  project  are  being  collected  both  at  the  University  of  Notre  Dame  and  Indiana 
University  at  South  Bend  to  gain  a  wider  range  of  abilities. 
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Progress 

We  have  completed  data  collection  for  two  studies  that  have  explored  the  accuracy  of 
traditional  measures  of  working  memory  and  how  well  they  predict  performance  on  various 
established  mental  model  measures.  Results  indicate  that  while  working  memory  measures  do 
predict  a  few  aspects  of  performance,  they  are  limited  in  their  ability  to  predict  future 
performance.  In  many  cases,  other  measures  of  complex  event  processing  were  better  predictors 
of  future  performance.  The  data  from  this  research  has  been  reported  in  four  papers.  One  is  in 
press  at  the  journal  Memory  &  Cognition  where  it  will  be  the  lead  article,  one  is  under  review  at 
the  Quarterly  Journal  of  Experimental  Psychology,  and  one  is  under  review  at  the  American 
Journal  of  Psychology.  The  fourth  manuscript  is  in  preparation. 

Currently,  we  are  collecting  data  for  a  series  of  experiments  aimed  at  investigating  how 
people  update  their  understanding  of  events.  In  particular,  our  focus  is  on  how  people  update 
spatial  and  temporal  information.  Specifically,  when  there  has  been  a  shift  in  space  or  time,  how 
efficient  are  people  at  making  this  shift?  Also,  when  people  have  problems  making  this  shift, 
what  is  the  nature  of  the  difficulty?  This  research  will  allow  us  to  gain  greater  insight  into  those 
aspects  of  comprehension  and  event  memory  that  are  difficult  for  people  versus  those  for  which 
people  show  a  fair  amount  of  competence.  This  information  will  allow  us  to  target  programs 
designed  to  increase  a  person’s  ability  to  comprehend  ongoing  information  about  events. 

Additionally,  we  are  continuing  to  develop  a  computer  simulation  of  mental  model 
construction  and  use.  To  constrain  its  initial  development,  we  have  limited  the  model  to  solving 
categorical  logic  problems.  To  date,  the  computer  model  has  been  successful  in  predicting  how 
accurately  and  quickly  people  are  able  to  understand  a  description  of  a  state  of  affairs  in  the 
world  and  draw  the  appropriate  inferences  from  that  information.  We  will  continue  to  expand 
this  model  to  contain  other  sorts  of  event  knowledge.  This  model  is  a  very  exciting  aspect  of  our 
research  because  it  has  the  potential  to  identify,  with  a  high  degree  of  precision,  the  specific 
nature  of  the  difficulties  people  have  in  constructing  mental  models  and  making  inferences  based 
on  their  understanding  of  the  world. 

Currently,  we  are  preparing  a  data  collection  for  a  study  which  will  investigate  the 
model’s  ability  to  identify  an  individual’s  comprehension  deficits,  create  a  training  program 
specifically  targeting  those  deficits,  and  assess  the  influence  of  that  training  both  immediately 
and  over  the  long  term.  This  proactive  approach  will  help  prevent  people  from  spending  time 
and  energy  training  on  cognitive  tasks  at  which  they  are  already  skilled.  Instead,  efforts  can  be 
targeted  to  an  individual’s  specific  deficiencies. 

Contributions  To  Basic  Science 

This  project  will  provide  us  with  a  more  advanced  understanding  of  working  memory 
capacity  and  processes  as  well  as  the  role  these  play  in  various  aspects  of  comprehension  and 
long-term  memory.  This  research  follows  and  integrates  long-time  traditions  in  each  of  these 
areas  of  cognition.  Furthermore,  a  better  understanding  will  be  gained  of  how  people 
comprehend,  monitor,  and  remember  changes  in  events  and  situations  in  the  world  based  on  the 
information  they  receive.  Lastly,  we  are  developing  a  more  precise  computer  model  of  event 
comprehension  and  memory  that  will  allow  for  a  much  more  specific  prediction  of  a  given 
individual’s  performance.  Because  we  are  interested  in  information  involving  complex  sets  of 
descriptions  about  events,  how  that  information  is  processed,  and  how  to  identify  people  who  are 
good  or  poor  at  this  sort  of  processing,  this  project  is  highly  relevant  for  the  U.S.  Army. 
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Potential  Applications 

The  potential  applications  of  this  research  for  the  Army  are  in  the  areas  of  training 
assessment  and  curriculum.  If  validated,  the  assessment  battery  measuring  complex  information 
comprehension  and  memory  may  be  used  to  identify  individuals  who  are  better  at  processing 
complex  information  and  potentially,  to  identify  an  individual’s  processing  weaknesses.  The 
training  guidelines  that  we  hope  to  develop  will  be  designed  to  target  those  aspects  of  event 
comprehension  and  memory  that  are  weak.  These  assessment  tools,  coupled  with  targeted 
training,  can  serve  to  minimize  the  amount  of  time  needed  to  increase  a  person’s  ability  to 
comprehend  and  remember  event  information. 
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Exploring  the  Interaction  of  Implicit  and  Explicit  Processes  to 
Facilitate  Individual  Skill  Learning 

Contract#:  DASW01-00-K-0012  Contract  dates:  9/00-8/03 

Institution:  University  of  Missouri-Columbia  PI:  Ron  Sun 

Co-PI:  Robert  Mathews 


Scientific  Objectives 

The  scientific  objectives  of  this  research  are  (1)  To  experimentally  investigate  the 
interaction  between  implicit  and  explicit  processes  in  skill  learning  (which  is  contrary  to  the 
common  tendency  of  studying  each  type  in  isolation)  with  the  goal  of  enhancing  learning 
through  knowledge  of  the  interaction  between  implicit  and  explicit  processes;  (2)  To  develop 
computational  models  and  theories  of  human  skill  learning  that  emphasize  the  interaction  of 
these  two  types  of  processes,  which  may  be  of  use  in  structuring  and  optimizing  training,  on  the 
basis  of  our  experimental  and  simulation  results. 

Approach 

The  current  work  develops  an  integrative  approach  for  skill  learning  that  takes  into 
account  both  implicit  and  explicit  processes.  Specifically,  we  focus  on  the  interaction  between 
the  two  types  of  processes  and  the  various  effects  of  this  interaction  on  improving  or  hampering 
learning  (e.g.,  the  synergy  effects;  Mathews  et  al  1989,  Sun  1997,  Sun  et  al  2001).  In  particular, 
we  will  focus  on  a  bottom-up  approach  (first  learning  implicit  knowledge  and  then  explicit 
knowledge  on  its  basis). 

A  computational  model,  CLARION,  which  is  markedly  different  from  other  existing 
cognitive  architectures,  will  be  further  developed  in  this  work  to  simulate  and  capture  a  range  of 
quantitative  data  that  are  related  to  the  interaction.  This  will  help  us  to  predict  (and  optimize) 
learning  processes.  We  will  carry  out  experiments  in  the  domain  of  process  control  tasks,  as  well 
as  the  domains  of  some  other  tasks,  and  generate  new  data  that  further  explicate  the  interaction 
between  implicit  and  explicit  processes.  These  anticipated  outcomes  (data,  models,  and  theories) 
will  provide  a  new  perspective  on  skill  learning.  Our  models  and  theories  will  be  useful  to  better 
understand  skill  learning  and  how  to  help  improve  learning  processes.  Our  models  and  theories 
may  also  be  useful  in  understanding  individual  differences  in  training  and  in  developing 
individualized,  adaptive  training. 


Progress 

We  further  developed  and  enhanced  the  basic  software  that  serves  as  our  paradigm  for  the 
experiments.  With  the  enhanced  software,  we  completed  4  experiments  and  designed  the  5th 
experiment  to  examine  reflection  with  different  types  of  hints.  Results  thus  far  suggest  that 
reflection  during  or  after  practice  on  this  task  has  little  benefit.  In  fact,  it  sometimes  interferes 
with  learning.  The  interference  effect  appears  to  stem  from  slowing  down  bottom-level  implicit 
learning  rather  than  a  direct  influence  of  top-level  learning  (generating  bad  explicit  rules). 
Evidence  for  this  comes  from  the  experiment  completed  last  semester  in  which  we  demonstrated 
that  reflecting  about  possible  rules  for  the  task  was  no  more  damaging  to  learning  than  simply 
predicting  the  next  output  level  or  even  doing  an  unrelated  task  (generating  sentences=20 
containing  specific  words).  Thus,  without  a  hint  about  how  to  reflect,  there  are  no  benefits  of 
explicit  thinking.  This  task  appears  to  be  learned  almost  completely  implicitly  when  no  hints  are 
provided.  This  is  interesting  given  that  learners  are  able  to  generate  some  valid  rules.  This 
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pattern  of  results  suggests  that  valid  rules  emerge  from  bottom-up  learning  rather  than  a  separate 
contribution  of  explicit  (top-level)  learning  when  no  hint  is  provided. 

Currently,  we  are  running  a  large  (16  cell)  experiment  exploring  reflection  with  different 
types  of  hints.  We  have  begun  collecting  this  data.  Additional  model  development  has  been 
accomplished.  We  enhanced  the  action  rule  mechanisms,  including  introducing  utility  and 
activation  measures.  The  enhancement  enables  the  calculation  of  response  times  in  the  model. 
We  are  also  introducing  non-action-centered  representations  to  the  model,  which  will  further 
enhance  the  capability  of  the  model.  New  simulation  has  been  carried  out  with  the  enhanced 
model. 

The  enhancement  enables  us  to  account  for  top-down  learning  (learning  that  goes  from 
explicit  to  implicit  knowledge),  in  addition  to  accounting  for  bottom-up  learning  (learning  that 
goes  from  implicit  to  explicit  knowledge).  We  simulated  some  new  human  data  sets  related  to 
skill  acquisition,  including  human  performance  in  learning  Tower  of  Hanoi,  letter  counting 
(alphabetic  arithmetic),  and  dynamic  process  control.  These  simulations  will  lead  to  new  insight 
into  skill  acquisition  processes. 

We  developed  a  framework  and  a  model  in  which  both  top-down  learning  (learning  that 
goes  from  explicit  to  implicit  knowledge)  and  bottom-up  learning  (learning  that  goes  from 
implicit  to  explicit  knowledge)  can  be  captured  in  a  rather  parsimonious  way.  Thus,  both  can  be 
explored  within  a  unified  framework  for  understanding  human  skill  acquisition. 

Ron  Sun  and  Bob  Mathews  co-organized  a  symposium  at  the  2001  Annual  Conference  of 
the  Cognitive  Science  Society,  on  the  interaction  of  implicit  and  explicit  processes  in  cognition, 
which  is  also  the  main  focus  of  our  project.  The  symposium  was  well  attended  and  received. 

Ron  Sun  published  one  conference  paper  on  preliminary  simulation  results  in  COGSCI2001. 

Ron  Sun  also  submitted  two  conference  papers  on  some  additional  simulation  results  to 
COGSCI’2002,  which  are  under  review  now.  Ron  Sun  published  one  journal  paper  in  Neural 
Networks  (co-authored  with  Antony  Browne),  another  one  in  Cognitive  Science  (co-authored 
with  Ed  Merrill  and  Todd  Peterson),  and  yet  another  in  Cognitive  Systems  Research.  He  also 
published  a  book  entitled:  “Duality  of  the  Mind." 

Contribution  to  Basic  Science 

Through  our  experiments,  we  will  further  explicate  the  interaction  between  implicit  and 
explicit  processes  in  cognition,  which  has  not  been  focused  on  by  previous  research  on  human 
cognition  (including  cognitive  skill  acquisition)  and  should  prove  to  be  of  significant  interest  to 
the  scientific  community.  Our  computational  model,  CLARION,  which  is  markedly  different 
from  other  existing  cognitive  architectures,  will  capture  a  wide  range  of  quantitative  data  that  are 
related  to  the  implicit/explicit  interaction,  which  will  be  of  significant  interest  to  the  cognitive 
science  community.  Overall,  our  research  results  will  provide  a  better  theoretical  perspective  on 
human  skill  learning.  In  particular,  our  models  and  theories  will  be  useful  in  improving  the 
theoretical  and  practical  understanding  of  individual  skill  learning  through  the  analysis  of  the 
interaction  between  implicit  and  explicit  processes. 
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Potential  Applications 

Our  anticipated  outcomes  (data,  models,  and  theories)  from  this  project  will  provide  a 
more  accurate  picture  of  human  skill  acquisition.  Our  models  and  theories  will  be  useful  in 
taking  into  consideration  individual  differences  in  training  and  in  developing  practical 
individualized  adaptive  training  methods.  The  theoretical  advances  will  translate  into  improved 
training  methods  in  a  wide  range  of  tasks.  This  project  has  just  started  and  the  application 
potentials  will  become  even  more  evident  as  the  project  progresses. 
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A  Program  for  Training  Sensemaking  Skills 

Contract  #:  1435-01-01-CT-31 161  Contract  Dates:  6/18/01-6/17/04 

Institution:  Klein  Associates  PI:  Gary  Klein 

Scientific  Objectives 

This  research  effort  is  comprised  of  three  technical  objectives: 

Objective  A:  Determine  the  nature  of  ‘sensemaking’  as  a  process  for  developing  situation 
awareness. 

Several  reseachers  have  offered  descriptions  and  models  of  situation  awareness.  Endsley 
(1995)  describes  three  levels  of  situation  awareness  and  suggests  metrics  for  each  level.  Tenney, 
Adams,  Pew,  Huggins,  and  Rogers  (1991)  and  Sarter  and  Woods  (1991)  offer  more 
contextualized  views  of  situation  awareness.  However,  to  our  knowledge,  no  effort  has 
undertaken  the  task  of  identifying  and  comprehending  the  process  by  which  individuals  develop 
situation  awareness  -  how  they  distinguish  between  relevant  and  irrelevant  information  and 
situational  cues,  draw  meaning  from  a  set  of  cues  offered  by  the  environment,  and  incorporate 
their  background  knowledge  into  an  assessment  of  the  current  situation.  This  is  an  attempt  to 
unpack  the  process  and  nature  of  situational  understanding  and  is  distinct  from  describing  or 
modeling  the  nature  of  situation  awareness  as  a  state.  We  are  employing  our  experience  studying 
higher-order  cognitive  functioning  in  naturalistic,  real-world  environments  to  unpack  the  process 
of  situational  understanding  by  using  subject-matter  experts  in  context. 

Objective  B:  Determine  the  effect  of  information  format  on  sensemaking. 

As  information  technologies  develop,  it  has  become  possible  to  present  information  in 
different  formats.  Where  information  once  was  shared  only  via  textual  reports  and  radio 
communications,  it  can  now  be  transmitted  as  photographs,  video  clips,  or  other  graphical 
representations.  Our  objective  is  to  determine  what  effect  cue  presentation  has  on  sensemaking. 
We  will  do  this  by  presenting  cues  and  information  in  two  different  formats.  In  the  first  version 
(presented  in  November  2001),  information  was  presented  using  text  and  a  map  in  paper-and- 
pencil  format.  The  second  version  will  be  enhanced  to  include  photographs,  video,  and  a 
timeline  presented  via  computer.  By  presenting  information  in  different  forms  and  comparing 
the  ability  of  the  participants  to  make  sense  of  the  information,  we  can  draw  conclusions  as  to  the 
effect  of  information  format  on  sensemaking.  Effects  will  be  examined  in  two  ways.  First, 
Cognitive  Task  Analysis  interviews  will  be  conducted  with  experts  and  novices  in  both 
presentation  modes.  This  will  allow  us  to  study  differences  in  expert  and  novice  strategies  and 
inferences.  Second,  we  will  also  train  novices  using  both  presentation  modes  to  determine  the 
effect  of  training  on  their  ability  to  develop  better  strategies  and  make  sharper  inferences. 

Objective  C:  Examine  techniques  for  hastening  individuals’  capabilities  to  comprehend 
ambiguous  situations. 

The  third  technical  objective  is  to  identify  training  interventions  that  can  increase  an 
individual’s  ability  to  make  sense  of  a  situation.  We  have  conducted  extensive  research  in  the 
area  of  naturalistic  decision  making  and  have  subsequently  developed  scenario-based  training 
interventions  intended  to  speed  the  acquisition  of  decision-making  expertise  (McCloskey,  Lake, 
Pliske,  &  Klein,  1998).  We  anticipate  experimenting  with  a  similar  technique  aimed  at  boosting 
sensemaking  skills.  This  methodology  is  called  FOCUS  -  Framework  for  Observing  and 
Comprehending  Uncertain  Situations. 
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FOCUS  is  conceptualized  as  follows:  A  situation  or  scenario  stands  at  the  center  of  the 
training.  The  student’s  goal  is  to  make  sense  of  the  situation  as  it  unfolds.  In  order  to  do  so, 
FOCUS  gradually  and  systematically  provides  pieces  of  information  about  the  situation  to  the 
student  -  these  may  be  situation  reports  or  Intel  reports  at  lower  levels  of  fidelity  or  video  clips 
and  imagery  at  higher  levels  of  fidelity.  The  student’s  task  is  to  integrate  the  information  into 
his/her  mental  picture.  We  believe  from  previous  research  that  experienced  operators  will  be 
able  to  make  sense  of  the  situation  with  fewer  pieces  of  information,  while  less-experienced 
individuals  will  take  longer  to  identify  what  is  happening.  The  training  intervention  enables 
students  to  practice  interpreting  a  situation  based  on  information  inputs  and  provides  them  with 
feedback  on  their  interpretations  via  a  window  into  experts’  thought  processes  as  they  conduct 
the  same  sensemaking  task.  By  comparing  one’s  assessment  of  the  situation  (and  reasoning 
behind  it)  at  various  points  in  time  to  that  of  one  or  more  experts,  the  trainee  can  identify  his/her 
development  needs  -how  to  draw  meaning  from  particular  pieces  of  information,  how  to  judge 
relevant  versus  irrelevant  information,  how  to  recognize  patterns  of  data  and  their  implications, 
and  how  to  recognize  and  clarify  anomalies  in  a  situation.  The  objective  is  to  experiment  with 
the  FOCUS  approach,  using  varied  levels  of  fidelity  and  different  techniques  for  providing 
feedback  to  students  using  the  sensemaking  process  of  the  experts. 

Approach 

Our  technical  approach  is  to  use  Situation  Awareness  Exercises  and  Cognitive  Task 
Analysis  to  develop  a  model  of  sensemaking  and  to  prepare  a  training  application  to  speed  the 
acquisition  of  sensemaking  skill. 

Situation  Awareness  Exercises  are  similar  to  Tactical  Decision  Games  used  by  the 
military  to  train  tactics  and  decision  making.  The  main  difference  is  that  situation  awareness 
scenarios  have  a  richer  description  of  cues  in  the  environment.  The  focus  is  on  understanding 
those  cues  and  patterns  of  cues  rather  than  on  making  a  decision.  The  meaning  of  these  cues  will 
not  be  immediately  apparent  and  may  only  make  sense  in  the  context  of  other  cues  or  by  mental 
simulation  to  put  oneself  in  the  shoes  of  the  enemy.  Some  of  the  cues  in  the  scenario  will  be 
meaningless  and  used  as  distractors. 

Situation  Awareness  Exercises  present  a  dilemma  in  an  unfolding  situation.  At  first  the 
situation  is  characterized  by  uncertainty  and  ambiguity.  As  the  situation  develops,  more  and 
more  information  is  provided,  but  the  situation  does  not  become  clear  until  the  participant  is  well 
into  the  situation.  By  the  last  situation  report,  everything  is  explained.  Because  of  the  unfolding 
nature  of  these  scenarios,  they  can  be  stopped  at  various  points  to  capture  “snapshots”  of 
participants’  situational  awareness  to  understand  how  they  currently  perceive  and  understand  the 
situation. 

Cognitive  Task  Analysis  is  the  process  of  understanding  the  cognitive  complexities  of  a 
task.  It  provides  a  set  of  tools  for  eliciting  and  representing  general  and  specific  knowledge 
pertaining  to  a  particular  activity,  in  this  case,  developing  an  accurate  assessment  of  a  situation 
within  the  Information  Operations  domain.  The  Cognitive  Task  Analysis  allows  us  to  go  beyond 
procedural  knowledge  and  behavioral  aspects  of  tasks.  The  purpose  is  to  get  inside  the  subject- 
matter  expert’s  head  and  understand  the  “cognitive  map”  or  mental  model  that  guides  his  or  her 
ability  to  make  sense  of  what  is  happening  in  a  complex  and  ambiguous  situation. 

We  are  conducting  Cognitive  Task  Analysis  interviews  with  both  novice  and  expert 
Information  Operations  officers  as  they  “play”  the  Situation  Awareness  Exercises.  We  will  elicit 
their  thought  processes  as  the  situation  unfolds  to  determine  how  they  are  interpreting  and 
drawing  inferences  from  cues  and  patterns  of  cues,  how  they  are  setting  expectancies,  what 
anomalies  they  spot  and  how  this  impacts  their  interpretation  of  the  situation,  and  how  they  use 
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their  mental  models  of  the  information  operations  domain  and  the  particular  context  (i.e.,  the 
mission,  and  the  cultural  and  political  environment)  in  order  to  build  and  update  their  situation 
awareness.  The  purpose  of  the  Cognitive  Task  Analysis  is  twofold.  First,  it  will  highlight  the 
differences  between  expert  and  novice  sensemaking  skills  and  processes  thereby  providing 
insights  as  we  articulate  a  model  of  the  sensemaking  process.  Second,  the  data  will  be  used  in 
training  to  show  less-experienced  individuals  the  meanings  and  inferences  that  highly 
experienced  operators  draw  from  the  pieces  of  information  within  the  Situation  Awareness 
Exercises. 


Progress 

To  date,  we  have  developed  three  paper-and-pencil  Situation  Awareness  Exercises.  All 
three  scenarios  use  the  U.S.  Army’s  current  mission  in  Bosnia  as  the  context  for  the  fictitious 
events.  We  conducted  the  exercises  coupled  with  the  Cognitive  Task  Analysis  interviews  at  the 
Land  Information  Warfare  Agency  (LIWA)  at  Fort  Belvoir,  VA.  Participants  included  6  expert 
and  6  novice  Information  Operations  officers.  We  are  well  into  the  process  of  analyzing  the 
data.  We  have  identified  several  types  of  inferences  participants  use  to  make  sense  of  the 
unfolding  situation  and  found  a  few  striking  differences  between  novice  and  expert  operators. 
The  data  have  led  us  to  drastically  alter  our  model  of  the  sensemaking  process,  and  we  are 
continuing  to  iterate  the  model  as  we  glean  more  meaning  from  the  data.  We  have  also  looked  to 
other  Klein  Associates’  data  sets  to  understand  and  articulate  the  sensemaking  process,  as  we 
seek  to  develop  a  model  that  is  not  specific  to  a  single  domain.  We  are  also  in  the  process  of 
refining  our  training  concept  based  on  our  experience  at  LIWA  and  the  sensemaking  model. 

Contributions  to  Basic  Science 

The  sensemaking  model  being  developed  in  this  effort  will  describe  a  higher-order 
cognitive  process  that  is  a  critical  component  of  decision  making  success  in  naturalistic 
environments.  Our  previous  research  offers  strong  evidence  that  in  naturalistic  settings 
characterized  by  uncertainty,  time  pressure,  and  high  stress,  good  decisions  are  a  result  of  good 
situation  awareness.  In  other  words,  once  an  individual  knows  what  type  of  situation  he  or  she  is 
dealing  with,  the  appropriate  decision  or  course  of  action  becomes  clear.  Therefore,  accurate 
sensemaking  is  requisite  for  good  decision  making.  This  effort  contributes  to  basic  science  by 
describing  this  critical  cognitive  process. 

Potential  Applications 

Defining  the  process  of  sense  making  opens  the  door  to  several  applications  for 
improving  sense-making  skills.  Structured  training  programs  that  replicate  the  process  of 
developing  expertise,  only  more  deliberately  and  at  a  faster  rate,  can  be  developed  and 
implemented  in  any  setting  that  requires  making  sense  of  a  situation  given  mounds  of  data  in 
various  formats.  In  addition,  toolsets  can  be  generated  for  individuals  to  apply  informally 
through  the  course  of  their  work  in  order  to  refine  their  sense  making  skills  and  strategies. 

Furthermore,  by  describing  how  humans  actually  use  information  to  develop  and  revise 
interpretations  of  a  situation,  we  can  develop  command-and-control  and  other  technologies  that 
work  in  concert  with  the  human  and  support  his/her  natural  process  for  filtering  information  and 
building  and  maintaining  situation  awareness.  There  may  also  be  implications  for  the  format  in 
which  certain  types  of  information  should  be  delivered,  which  will  give  an  individual  the  best 
chance  at  interpreting  the  situation  accurately. 
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RACO  RESEARCH  OBJECTIVE  #2: 


Provide  fundamental  knowledge  to  improve  leader  assessment  and 
accelerate  leader  development. 


Research  that  falls  under  this  heading  provides  concepts  and  methods 
for  accelerating  leader  development,  understanding  and  developing 
leader  adaptability  and  flexibility,  and  discovering  and  testing  the 
basic  cognitive  principles  that  underlie  effective  leader-team 
performance.  Understanding  the  dynamics  of  small  group  leadership 
in  face-to-face  and  distributed  team  environments  is  critical  to  this 
research  objective  as  well. 
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Learning  Leadership  Skills  in  Distributed  Training  Scenarios: 
Diagnosing  Strategies  in  Scenarios  Using  Latent  Semantic  Analysis 

Contract  #:  DASW01-00-K-0017  Contract  Dates:  9/1/00-8/31/02 

Institution:  George  Mason  Univ/Univ  of  Pittsburgh  PI(s):  Christian  Schunn 

Scientific  Objectives 

The  primary  objectives  of  this  research  are  to  (a)  test  a  theoretical  model  of  expertise  in 
complex  tacit  knowledge  domains  like  leadership  and  (b)  test  a  diagnosis  and  training  approach 
that  is  based  on  that  theoretical  model  of  expertise. 

The  theoretical  model  being  applied  to  leadership  assumes  that  one  important  component 
of  expertise  is  the  effective  and  appropriate  use  of  strategies.  Expert  strategy  use  generally 
differs  from  novice  strategy  use  along  four  dimensions  (Lemaire  &  Siegler,  1995).  In  order  to 
provide  appropriate  training  to  a  novice,  one  has  to  diagnose  the  particular  type  of  strategy  use 
problems  exhibited  by  the  given  novice.  The  particular  problems  include  (a)  missing  strategies 
entirely,  (b)  under-using  generally  effective  strategies,  (c)  poorly  executing  strategies,  and  (d) 
executing  good  strategies  at  the  wrong  times. 

Diagnosing  strategy  use  in  complex  domains  like  leadership  presents  a  difficult 
challenge.  Thus,  the  secondary  objectives  of  this  research  are  to  explore  mechanisms  for 
automatically  diagnosing  strategy  use  and  larger  scale  problems  in  patterns  of  strategy  use  (along 
the  4  described  dimensions)  in  complex  domains,  particularly  the  strategies  found  in  written 
responses.  If  such  a  method  can  be  developed,  then  this  strategies  training  approach  can  be 
effectively  applied  in  distributed,  distance  training  situations. 

Approach 

I  proposed  a  new  mechanism  by  which  individuals  can  acquire  training  on  leadership 
skills  in  distributed  training  situations.  The  first  step  is  to  construct  a  moderately  large  set  of 
leadership  scenarios  to  which  a  trainee  must  respond.  I  selected  platoon  leadership  as  the 
particular  domain.  The  second  step  is  to  identify  a  set  of  common,  generic  strategies  that  could 
be  applied  to  each  of  the  scenarios.  This  step  is  implemented  through  a  mixture  of  the  top-down 
application  of  existing  leadership  literatures  and  the  bottom-up  analysis  of  similarities  in  ratings 
of  particular  possible  responses  by  individuals  across  scenarios.  Third,  I  proposed  to  instantiate 
each  of  those  strategies  for  each  scenario,  producing  strategy-scenario-ideals.  Fourth,  I  will  use 
a  new  knowledge  representation  mechanism  embodied  in  Latent  Semantic  Analysis  (Landauer  & 
Dumais,  1997)  to  automatically  capture  strategy  use  in  written  responses  to  the  leadership 
scenarios.  The  approach  is  to  compare  written  responses  for  a  scenario  to  the  strategy  ideals  for 
that  scenario. 

The  resulting  set  of  information  will  be  used  to  develop  a  computerized  training 
environment  for  training  platoon  leadership  skills.  The  computerized  training  environment  will: 
(1)  automatically  grade  the  quality  of  written  responses  to  the  Platoon  Leadership  scenarios;  (2) 
automatically  categorize  the  strategies  used  in  the  written  responses  to  the  scenarios;  (3) 
automatically  diagnose  a  variety  of  problems  in  the  strategy  use;  and  (4)  provide  relevant 
training  feedback  to  address  the  diagnosed  problems.  The  components  and  the  entire  training 
environment  will  be  empirically  tested  using  ROTC  Army  cadets  at  several  stages  of 
development  of  the  environment. 
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Progress 

To  develop  a  base  set  of  diagnostic  and  training  examples,  we  have  expanded  the  original 
set  of  15  platoon  leadership  scenarios  in  the  TKML  (Tacit  Knowledge  Military  Leadership)  to  50 
scenarios.  These  50  scenarios  cover  the  main  types  of  skills  typically  associated  with  platoon 
leadership.  The  new  scenarios  were  adapted  from  scenarios  presented  in  the  Army  Leadership 
Tacit  Knowledge  Corpus.  Because  these  scenarios  are  to  be  used  for  written  responses  and  our 
goal  is  to  diagnose  particular  leadership  strategies  in  the  responses,  the  scenarios  were  designed 
to  require  a  single  strategy  response  rather  than  a  long  list  of  different  strategies. 

In  preparation  for  the  LSA  phase  of  the  project,  we  acquired  the  specialized  computer 
and  software  to  run  LSA  and  successfully  conducted  basic  test  runs.  We  then  tested  LSA  using 
various  military  leadership  databases  as  input,  including  the  vignettes  from  the  Army  Leadership 
Tacit  Knowledge  Corpus  and  the  Army  Military  Leadership  Field  Manual  FM  22-100. 

We  collected  data  from  25  ROTC  cadets,  with  each  cadet  generating  text  responses  to  at 
least  10  open-ended  scenarios.  We  developed  a  set  of  13  leadership  strategies  based  on  cards 
sorts  of  possible  options  in  the  Platoon  TKML  ratings  task,  and  then  conducted  some  testing  of 
these  strategies  on  the  open-ended  text  data. 

To  develop  an  automatic  method  for  coding  strategies  in  the  open-ended  text  responses  to 
be  used  in  the  intelligent  tutoring  system,  we  went  through  a  number  of  steps.  First,  a  research 
assistant  generated  an  example  of  how  to  implement  each  of  the  13  strategies  in  each  of  the  10 
scenarios  (selecting  only  10  as  a  trial  set).  These  implementations  were  called  scenario-strategy- 
ideals.  Then  LSA  compared  each  open-ended  response  to  these  scenario-strategy-ideals  and 
assigned  the  response  to  the  strategy  category  with  the  best  matching  ideal.  This  method  proved 
to  be  better  than  chance,  but  well  below  human  coding  accuracy  and  well  below  usable  levels  for 
later  use  in  intelligent  tutoring. 

Through  a  gradual  process  of  refinement  in  coding  actual  open-ended  responses  using 
these  13  strategies,  we  narrowed  the  set  down  to  10  general  strategies  that  could  be  coded  more 
reliably.  We  then  asked  a  different  set  of  28  ROTC  cadets  to  generate  implementatons  of  each  of 
the  strategies  in  each  of  the  scenarios  (5  strategies  for  5  scenarios  for  each  given  cadet).  Because 
there  appeared  to  be  multiple  legitimate  ways  to  implement  a  given  strategy  in  a  given  scenario, 
we  wanted  data  on  the  variety  of  ways  this  could  be  acheived.  Coders  carefully  went  through 
this  forced  strategy  data  to  make  sure  that  the  cadets  did  indeed  respond  with  the  given  strategy 
in  the  given  scenerio  context.  Only  a  small  proportion  of  the  forced  data  was  removed  due  to  not 
implementing  the  indicated  strategy. 

With  the  forced  strategy  data  in  hand,  we  then  reconducted  the  LSA  analyses.  This  time 
the  LSA  strategy  coding  results  were  close  to  human  coding  accuracy — approximately  50% 
aggreement.  Our  next  steps  will  be  to  use  a  much  richer  text  input  and  improve  the  matching 
algorithm.  The  current  text  input  to  LSA  for  generating  its  semantic  space  was  much  smaller 
than  is  typically  used  in  successful  applications  of  LSA.  The  current  classification  algorithm  of 
nearest  exemplar  is  one  very  simple  classification  approach  out  of  a  myriad  of  currently  existing 
classification  algorithms.  We  will  explore,  for  example,  using  a  Bayesian  approach  that  also 
takes  into  account  secondary  matches. 

To  study  the  effects  of  expertise  on  these  strategies,  we  also  collected  data  on  the  open- 
ended  task  from  undergraduates  completely  unfamiliar  with  either  leadership  or  the  Army.  We 
will  be  collecting  comparable  data  from  experts  (Its,  capts,  majors)  this  summer  from  two 
military  bases  as  part  of  Umbrella  week  activities.  Coding  of  all  this  data  will  be  complete  by 
mid-July  and  analyses  of  expertise  effects  should  be  complete  by  the  end  of  August.  We  have 
already  developed  and  tested  the  mathematical  analyses  that  will  be  applied  to  measure  expertise 
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effects  along  each  of  the  4  dimensions  of  strategy  expertise  as  outlined  by  the  model  we  are 
testing. 

We  have  sketched  out  the  form  of  an  intelligent  tutoring  system  that  would  use  this 
automated  classification  system.  We  will  implement  and  test  a  prototype  of  the  system  this 
summer.  Until  the  classification  algorithm  approaches  near  85%  accuracy,  however,  there  is  no 
point  in  fully  implementing  the  tutoring  system. 

Contributions  to  Basic  Science 

The  central  theoretical  goals  of  this  project  will  be  to  provide  a  strong  test  of  theories  of 
expertise  and  knowledge  representation  for  ill-structured  domains  like  leadership.  In  particular, 
the  data  collected  in  this  project  will  be  used  to  test  and  refine  theories  of  strategy  selection, 
expertise,  and  knowledge  representation  as  they  apply  to  complex,  ill-structured  domains  like 
leadership. 


Potential  Applications 

The  application  of  our  intelligent  tutoring  work  is  clear:  if  the  automatic  strategy 
identification  methodology  works  and  we  can  build  a  successful  tutoring  system.  By  using  it, 
then,  we  will  have  developed  a  powerful  methodology  for  intelligent  tutoring  in  many  complex, 
ill-defined  domains. 

We  also  expect  to  learn  more  about  what  common  strategies  can  be  effectively  applied 
across  a  wide  range  of  platoon  leadership  situations.  This  knowledge  in  itself  could  also  be 
applied  to  other,  more  traditional  training  methods  beyond  the  particular  automated  method 
being  pursued  here. 


24 


Interactivity,  Communication,  and  Trust: 

Challenges  and  Opportunities  for  Leadership  in  the  Electronic  Age 

Contract  #:  DASW01-98-K-0009  Contract  Dates:  9/15/98-3/13/01 

Institution:  University  of  Arizona  PI:  Judee  K.  Burgoon 

Co  Pi’s:  Suzanne  Weisband  & 
Joseph  Bonito 


Scientific  Objectives 

The  rapid  and  pervasive  diffusion  of  new  communication  technologies,  coupled  with 
increased  emphasis  on  distributed  teams  and  teamwork,  is  fundamentally  transforming  the 
modem  military.  Guided  by  the  principle  of  inter-activity,  two  related  research  programs  are 
examining  the  effect  of  these  technologies  on  leadership,  team  performance,  and  related 
communication  processes  and  outcomes.  Research  questions  being  addressed  include: 

1.  What  structural  properties  of  communication  platforms  or  arrangements  “afford”  or 
prevent  interactivity?  What  communication  processes  are  experienced  as  interactive? 

2.  How  do  structural  properties  such  as  mediated/nonmediated  and  co-located  versus 
distributed  communication  affect  the  quality  of  team  communication,  social 
judgments  (e.g.,  trust)  and  task  performance?  Which  arrangements  are  best  able  to 
establish  and  maintain  trust  when  team  members  do  not  interact  face-to-face  (FtF)? 
Which  properties  need  to  be  retained  or  augmented  to  maximize  leadership  and 
mitigate  effects  of  heavy  task  load? 

3.  Are  the  effects  of  using  electronic  forms  of  communication  transitory  or  lasting? 

4.  How  should  leaders  be  trained  to  motivate  distant  team  members  communicating 
electronically  to  perform  their  best  and  to  raise  the  team's  collective  success  in 
accomplishing  their  task? 


Approach 

Experimental  Interactivity  Research  Program 

In  light  of  initial  findings  from  pilot  work  and  our  previous  research,  and  given  the 
continuation  contract  we  have  been  awarded,  we  have  modified  and  expanded  the  original  scope 
of  work  to  include  five  experiments. 

Experiment  1  tests  the  impact  of  user  participation.  Pairs  of  strangers  collaborate  on  a 
decision-making  task  that  is  videotaped  and  subsequently  viewed  by  observers.  Observer 
perceptions  and  performance  will  then  compared  to  participant  perceptions  and  performance. 

Experiment  2  tests  the  effects  of  two  additional  structural  properties,  mediation  and 
geographic  proximity,  by  comparing  FtF  participants  to  those  who  engage  in  proximal  text-based 
interaction  and  those  who  engage  in  distributed  text-based  interaction. 

Experiment  3  tests  modality  and  context  richness  by  comparing  FtF  interaction  to  text, 
audio,  and  audiovisual  interaction.  In  all  cases,  participants  perform  a  decision-making  task. 
After  this  task  they  complete  3  sets  of  measures:  (1)  process-related  dependent  measures 
(assessments  of  communication  involvement,  degree  of  coordination,  mutuality,  understanding, 
and  perceived  similarity  among  participants,  and  ease  of  interaction),  (2)  social  judgments  about 
their  partner  (a  multidimensional  measure  of  trust,  credibility,  and  attraction  toward  the  partner 
and  a  measure  of  the  expectedness  of  the  partner’s  behavior),  and  (3)  outcome-related  dependent 
measures  (influence,  quality  of  decision  making,  accuracy  of  information  recall). 
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Experiment  4  has  now  become  a  longitudinal  version  of  Experiment  3.  It  incorporates 
three  different  tasks  (a  social  task,  a  case  study  decision-making  task  designed  by  Moberg,  and 
the  Desert  Survival  Problem)  across  two  time  periods. 

Lastly,  experiment  5  examines  synchronicity  by  comparing  processes  and  outcomes 
under  conditions  of  synchronous  (real-time)  and  asynchronous  interaction.  This  research  is  part 
of  a  cooperative  effort  with  a  large-scale  project  testing  strategies  for  achieving  smoking 
cessation.  Groups  perform  several  tasks  using  Group  Systems  (a  computerized  group-support 
tool)  under  one  of  three  conditions:  (1)  FtF,  synchronous,  (2)  distributed,  synchronous,  or  (3) 
distributed,  asynchronous  (delayed  time).  Group  members  conduct  electronic  brainstorming, 
organize  concepts,  ballot  and  prioritize  strategies  all  via  computer.  This  experiment  permits  the 
separation  of  the  effects  of  working  at  a  distance  from  the  effects  of  interacting  at  the  same  or 
different  times. 

Team  Development  Research  Program 

The  other  prong  of  our  research  employs  a  field  study  in  which  four-person  teams  of 
graduate  and  undergraduate  students  in  two  geographically  distant  U.S.  universities  participate  in 
virtual  team  projects  developed  for  their  MIS  classes.  The  project  simulates  work  in  temporary 
virtual  teams  in  that  it  (1)  lasts  4  weeks,  (2)  introduces  high  vulnerability  because  the 
interdependent  task  requires  team  interactions  to  achieve  objectives,  and  (3)  entails  high 
uncertainty  as  everyone  is  a  stranger  and  all  communications  are  conducted  using  mediated 
communication.  To  analyze  strategies  for  interaction,  we  focused  on  the  actual  communication 
in  context.  Strategies  associated  with  initiations  and  responses  were  also  analyzed  as  way  of 
measuring  trust.  To  initiate  interaction  requires  trust  because  it  places  the  initiator  in  a  position 
of  vulnerability.  The  making  of  responses  and  signals  inspires  trust  and  belief  that  the  group  is 
responsible  and  skillful  enough  to  handle  the  uncertainties  in  front  of  them.  In  addition  to  the 
electronic  messages,  we  collected  data  from  pre-  and  post-project  questionnaires.  These 
questionnaires  measured  demographics,  school  performance,  computer  access,  team  and 
electronic  communication  experience,  and  perceptions  of  leadership.  Team  performance  was 
assessed  by  the  team  grade,  which  was  mutually  determined  by  professors  in  both  universities. 

One  unanticipated  addition  to  our  field  research  effort  was  participating  in  the  Rim  of  the 
Pacific  Strong  Angel  2000  humanitarian  assistance/disaster  relief  exercise  that  was  conducted 
jointly  by  the  United  Nations,  the  Third  Fleet,  representatives  of  all  branches  of  the  U.  S.  Armed 
Forces,  and  six  other  cooperating  nations.  This  massive  simulation  of  hostilities  between 
neighboring  countries  included  the  creation  of  a  camp  for  fleeing  refugees.  This  provided  an 
ideal  “field  laboratory”  for  examining  ways  in  which  collaborative  communication  and 
coordination  could  be  established  among  representatives  with  diverse  backgrounds  and  goals 
under  highly  adverse  conditions  in  an  austere  environment.  Results  from  this  onsite 
observational  project  will  be  included  in  the  continuation  contract. 

Progress 

We  have  now  collected  all  data  for  all  the  experiments.  We  have  also  conducted  pilot 
tests  and  started  to  collect  data  for  the  continuation  contract.  We  are  beginning  our  analysis  of 
the  data  for  Experiment  4  (longitudinal  richness)  and  Experiment  5  (synchronicity).  We  are 
preparing  manuscripts  for  conference  submission  and  publication,  including  an  invited  article  for 
the  Journal  of  Communication.  Other  work  has  now  been  published  or  is  in  press  in  Journal  of 
Management  Information  Systems.  Computers  and  Human  Behavior,  and  Human 
Communication  Research.  Additionally,  we  have  also  started  to  analyze  the  field  data  collected 
as  part  of  the  RIMPAC  Strong  Angel  2000  exercise  as  part  of  the  continuation  contract. 


26 


Contributions  to  Basic  Science 


Interactivity  Research  Program 

The  results  of  this  research  have  direct  implications  for  current  organizational  theories 
that  address  distributed  "virtual"  processes  for  accomplishing  work.  As  work  becomes  more 
global  and  distributed,  the  whole  nature  of  organizing  will  inevitably  be  challenged  and 
modified.  Such  changes  need  to  be  guided  by  systematic  empirical  work  rather  than  by  anecdote 
and  personal  experience,  which  in  many  quarters  form  the  primary  source  of  “data. 

Moreover,  the  longitudinal  experimental  and  field  work  closely  resemble  the  kinds  of 
work  contexts  faced  by  members  of  the  same  organization  and  members  of  teams  assigned  to 
projects  with  distant  others.  Therefore,  the  results  should  have  greater  generalizability  than 
many  prior  investigations. 

Team  Development  Research  Program 

The  results  of  this  study  provide  evidence  that  frequent  and  explicit  awareness 
communication  improves  team  performance.  In  particular,  it  is  early  team  interaction  that 
contributes  most  to  successful  team  outcomes.  We  also  found  that  project  leaders  develop  the 
interaction  patterns  necessary  for  team  success.  The  implication  is  that  when  leaders  model  and 
promote  the  exchange  of  task  information  and  awareness  of  others,  the  team  members  may  be 
more  motivated  to  work  hard  on  behalf  of  the  team. 

Potential  Applications 

Interactivity  Research  Program 

Our  studies  clearly  indicate  that  for  a  given  meeting,  project,  or  ongoing  task,  the  military 
must  first  determine  the  kinds  of  process  and  outcome  goals  that  are  required,  then  select  or 
modify  CISs  to  match  task  objectives  with  the  appropriate  means  of  communication.  Our 
research  indicates  that  the  format  selected  can  facilitate  or  undermine  desired  outcomes  by 
eliciting  very  different  kinds  of  communication  among  leaders  and  team  members.  The  degree 
of  interactivity  desired  must  be  considered  in  advance.  Protocols  should  be  developed  that 
specify  the  “best  practices”  in  terms  of  matching  communication  and  information  tools  to  desired 
objectives.  Such  protocols  can  also  identify  potential  pitfalls  to  avoid  or  compensatory  efforts 
that  can  be  used  so  that  negative  consequences  such  as  noncompliance,  distrust  and 
miscommunication  do  not  occur. 

Team  Development  Research  Program 

Our  field  research  has  uncovered  some  features  of  communication  that  are  needed  to 
create  and  sustain  trust  so  that  distant  teams  and  leaders  can  be  successful.  A  natural  outgrowth 
of  this  work  should  be  the  development  of  training  procedures  for  leaders  in  mediated  settings. 
Other  potential  applications  are  the  development  of  software  that  will  create  a  more  visible  form 
of  social  awareness  for  team  members  and  leaders  who  cannot  communicate  face-to-face. 
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The  Leadership  Formula:  P*M*D 

Contract  #:  DASW01-01-K-0004  Contract  Dates:  09/01/01-03/01/04 

Institution:  The  Center  for  Outstanding  Leadership  PI(s):  Reuven  Gal  &  Micha  Popper 

Scientific  Objectives 

The  present  study  examines  the  leadership  development  process  through  a  field  study  in 
the  Israeli  Defense  Force  (IDF).  The  conceptual  framework  suggests  that  leadership  is  a 
function  of  a  given  potential  (P),  a  relevant  motivation  (M)  and  continuing  development  (D). 
These  three  components  comprise  the  “formula”  (P*M*D)  for  outstanding  leadership.  All  three 
components  are  necessary  prerequisites  for  actual  effective  leadership. 

This  “formula”  and  the  theoretical  framework  behind  it  extend  the  leadership  literature  by 
emphasizing  internal  aspects  (e.g.,  personality  predispositions,  motivational  needs  and 
developmental  processes)  that  give  rise  to  the  potential  to  become  and  the  motivation  to  assume 
leadership  positions.  Our  argument  is  that  only  those  who  have  sufficient  “ego  resources”  (self 
efficacy,  internal  locus  of  control,  low  level  of  anxiety,  secure  attachment  pattern  and  tendency 
for  optimism),  are  highly  motivated  to  lead,  and  are  “open”  to  personal  development  can  become 
leaders. 

Guided  by  this  conceptual  framework,  our  research  hypotheses  are  as  follows: 

1.  ‘Leaders’  will  report  higher  level  of  self  efficacy  compared  to  “non-leaders.’ 

2.  ‘Leaders  will  exhibit  a  stronger  internal  locus-of-control  than  ‘non-leaders.’ 

3.  ‘Leaders’  will  demonstrate  higher  level  of  optimism  than  ‘non-leaders.’ 

4.  ‘Leaders’  will  be  characterized  by  lower  level  of  trait-anxiety  than  ‘non-leaders.’ 

5.  ‘Leaders’  will  be  characterized  by  a  ‘secure’  or  ‘avoidance’  attachment  pattern, 

whereas  ‘non-leaders’  will  be  characterized  by  an  ‘anxious’  attachment  pattern. 

6.  ‘Leaders’  will  have  a  higher  level  of  motivation  to  lead  than  ‘non-leaders.’ 

7.  ‘Leaders  will  report  a  greater  number  of  and  more  powerful  ‘leadership-shaping’ 
experiences  accrued  in  their  early  childhood  and  adolescence  periods  than 
‘non-leaders’. 


Approach  &  Progress 

This  study  is  based  on  a  comparative  study  of  several  samples  of  IDF  soldiers  (from 
Infantry  and  Armour)  undergoing  basic  training.  By  distinguishing  “leaders”  from  “non-leaders” 
and  subsequently  assessing  various  P,  M  and  D  measurements,  this  field  study  will  enable  the 
examination  of  our  three  categories  of  hypotheses  related  to  the  leadership  formula:  potential, 
motivation  and  development. 

In  order  to  test  our  research  tools  (especially  the  new  tools  we  have  constructed)  and  to 
test  the  research  procedures  (working  with  army  units),  we  conducted  a  pilot  study.  This  pilot 
study  was  conducted  during  the  months  of  February  and  March  2002.  Participants  included  195 
soldiers  from  Infantry  and  Armor  units.  We  have  started  to  analyze  this  data.  Additionally,  we 
returned  to  the  same  soldiers  a  few  weeks  back  (May  2002)  in  order  to  test  the  reliability  of  the 
new  questionnaires  we  developed. 

So  far,  the  findings  concerning  the  potential  and  motivational  variables  seem  to  support 
our  hypotheses.  Currently,  we  are  examining  the  developmental  variables  by  analyzing  the  data 
obtained  from  LSE  interviews  and  questionnaires. 
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Contributions  to  Basic  Science 

The  results  obtained  from  the  pilot  study  support  our  ‘potential’  and  ‘motivational’ 
hypotheses.  To  become  a  leader,  one  should  have  (a)  enough  “ego  resources”  (self-efficacy, 
internal  locus  of  control,  low  level  of  anxiety,  secure  attachment  pattern  and  tendency  for 
optimism),  and  b)  a  strong  motivation  to  lead  others.  An  examination  of  the  developmental 
issues  is  currently  underway.  After  completing  this  final  analysis,  we  will  test  our  leadership 
formula  using  the  pilot  data. 

It  is  important  to  understand  that  the  actual  field  study  (that  will  start  in  September  2002) 
will  further  examine  our  hypotheses,  especially  the  ones  that  focus  on  development.  Our  main 
scientific  contribution  thus  far  (up  to  this  research  stage)  has  been  in  forming  and  constructing 
new  leadership  tools.  We  have  expanded  the  “motivation-to-lead’  concept,  both  substantially 
and  technically.  We  have  prepared  three  developmental  leadership  tools:  1)  an  openness  to 
experience  questionnaire,  2)  the  LSE  interview  and  3)  an  LSE  questionnaire.  Finally,  we  have 
formed  a  questionnaire  to  identify  leaders  and  non-leaders. 

Potential  Applications 

This  research  is  being  conducted  in  the  Israeli  Defense  Force  (IDF).  Unlike  civilian 
organizations,  in  which  leadership  is  diffused  in  the  manager’s  functions  and  formal  authority,  in 
the  military  ‘net  leadership’  is  much  more  prominent.  In  fact,  leadership  is  the  essential  driving 
force  that  affects  a  soldier’s  readiness  to  risk  his/her  life.  Furthermore,  within  the  military,  the 
distinction  between  ‘leaders’  and  ‘non-leaders’  is  much  more  apparent  than  in  most  other 
institutions.  However,  this  distinction  is  somewhat  contaminated  in  those  military  organizations 
(e.g.,  U.S  Army)  in  which  the  differentiation  between  officers  and  non-officers  is  done 
‘institutionally’  (i.e.  through  military  academies)  rather  than  ‘naturally’  (namely,  through  a 
selection  process). 

A  unique  example  of  the  latter  type  occurs  in  the  IDF.  Starting  on  equal  footing,  all 
Israeli  conscripts  are  gradually  undergoing  an  ongoing  selection  process  during  their  initial 
training  period.  This  unique  situation  makes  the  IDF  an  extraordinary  natural  laboratory  to 
examine  leadership  development.  Moreover,  this  field  study  is  being  conducted  on  two  different 
army  units  (Infantry  and  Armor)  in  order  to  examine  whether  they  undergo  different  leadership¬ 
forming  processes. 

Overall,  this  study  provides  a  unique  opportunity  to  understand  how  ‘leaders’  differ  from 
‘non-leaders,’  what  makes  ‘net-leadership,’  and  how  it  develops.  Furthermore,  our  research  may 
unravel  some  of  the  mysteries  related  to  the  nature-nurture  origins  of  leadership  as  well  as 
suggest  some  important  practical  implications  related  to  leader  selection  and  development  in  any 
modem  military. 
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Developing  Effective  Military  Leaders: 

Facilitating  the  Acquisition  of  Experience-Based,  Tacit  Knowledge 

Contract  #:  DASW01-00-K-0014  Contract  Dates:  9/01/00  -  9/30/03 

Institution:  Yale  University  PI(s):  Robert  J.  Sternberg 


Scientific  Objectives 

The  purpose  of  our  recent  research  efforts  is  twofold.  First,  we  seek  to  deepen  our 
understanding  of  the  cognitive  processes  occurring  during  the  acquisition  and  application  of  tacit 
knowledge  (TK)  to  solving  practical  problems.  Second,  we  seek  to  establish  that  training 
interventions  can  enhance  practical  problem-solving  skills  and  the  ability  to  acquire  TK  from 
experience.  This  work  represents  an  extension  of  previous  research  indicating  a  relation  between 
TK  and  occupational  performance  in  that  (a)  we  have  begun  to  characterize  the  underlying 
processes  through  which  TK  aids  in  problem  solving,  and  (b)  we  are  using  our  knowledge  of 
these  processes  to  understand  how  they  may  be  facilitated  through  training. 

Approach 

Our  approach  to  illuminating  the  cognitive  processes  underlying  the  acquisition  and 
application  of  TK  to  practical -problem  solving  involves  the  development  of  extended  case 
studies  that  reflect  the  use  of  metacognitive  problem-solving  processes.  In  contrast  to  measuring 
the  outcome  of  having  TK  through  quality  ratings  of  problem-solving  strategies,  the  extended 
case  studies  allow  measurement  of  cognitive  processes  involved  in  applying  TK  through  the  use 
of  open-ended  questions  that  deal  with  (a)  problem  identification,  (b)  solution  generation,  and  (c) 
outcome  monitoring.  We  have  examined  (a)  the  measurement  properties  of  these  three 
cognitive-processing  factors,  which  appear  to  be  sound,  and  (b)  the  relationship  between 
officers’  scores  on  these  factors  and  their  scores  on  the  Tacit  Knowledge  Inventory  for  Military 
Leaders  (TKML),  in  which  some  interesting  patterns  have  been  found. 

Our  approach  to  enhancing  TK  acquisition  involves  designing  competing  interventions  to 
facilitate  the  acquisition  of  TK,  each  of  which  emphasizes  a  different  aspect  of  TK.  One 
intervention  will  emphasize  the  condition  aspect  of  TK,  drawing  attention  to  the  conditions  that 
are  identified  as  triggers  for  action.  Another  intervention  will  emphasize  the  action  aspect  of  TK, 
providing  methods  for  reflecting  on  action  outcomes.  A  third  intervention  will  target  both  the 
condition  and  action  aspects  of  TK.  Each  intervention  is  designed  to  provide  leaders  with  the 
skills  needed  to  regulate  their  own  learning.  We  will  compare  the  outcomes  of  each  intervention 
to  each  other,  as  well  as  to  a  non-intervention  control  condition.  We  will  test  the  effectiveness  of 
each  intervention  on  learning  and  performance  outcomes  at  varying  levels  of  command. 

Progress 

To  date,  our  research  regarding  the  cognitive  processes  underlying  the  application  of  TK 
to  practical  problem  solving  have  produced  promising  results.  Specifically,  using  confirmatory 
factor  analysis  we  have  made  the  following  key  discoveries:  (a)  our  measures  of  cognitive 
processes  (i.e.,  the  case  studies)  are  reliable  and  stable,  (b)  performance  on  these  measures  could 
be  best  accounted  for  by  three  metacognitive  problem-solving  factors  comprising  problem 
identification,  solution  generation,  and  outcome  monitoring,  and  (c)  the  relation  between 
problem  identification  and  TK  scores  was  strongest  of  all  the  problem-solving  factors, 
independent  of  rank.  These  findings  are  currently  being  prepared  for  publication. 
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In  the  past  year,  we  have  restructured  the  design  of  our  TK  training  interventions  relative 
to  our  initial  proposal,  based  on  a  completed  literature  review  and  our  latest  findings.  In 
addition,  because  of  changes  in  access  to  active-duty  military  leaders  (as  a  result  of  the  terrorist 
attacks  on  September  1 1th),  we  have  worked  to  gain  inroads  to  other  military  personnel  to 
participate  in  our  research.  We  will  be  working  closely  with  Army  personnel  and  reviewing 
Army  documentation  to  structure  our  instructional  materials  for  maximum  face  validity  and 
benefit  to  military  participants.  In  particular,  we  will  be  working  with  LTC  Jim  Rice,  MAJ 
Benjamin  Webb,  and  the  soldiers  of  1st  Battalion,  1st  Infantry  at  the  U.S.  Military  Academy. 

We  have  drafted  (1)  preliminary  versions  of  our  instructional  materials,  which  include  an  outline 
of  the  instructional  goals  and  learning  outcomes  of  the  intervention,  (2)  a  set  of  discussion 
questions  based  on  the  case  study  that  will  be  administered  as  part  of  the  TK  pre-tests,  and  (3) 
response  data  from  expert  leaders  indicating  the  “correct”  answers  to  questions  on  the  case  study 
and  (4)  predictions  for  the  outcome  of  particular  alternative  problem  solutions. 

Contributions  to  Basic  Science 

A  theory  that  attempts  to  explain  human  functioning  in  everyday  life  is  by  necessity 
complex  and  multifaceted.  Testing  and  improving  this  theory  is  a  long  process,  in  which  the 
theory  is  addressed  piece  by  piece.  The  major  contribution  of  the  current  work  to  basic  science 
is  the  illumination  of  a  previously  unexplored  piece  of  the  triarchic  theory  of  intelligence.  This 
piece  describes  the  knowledge-acquisition  components,  the  metacognitive  problem-solving 
components,  and  the  performance  components  of  practical  intelligence  and  the  role  each  plays  in 
TK  acquisition  and  application.  Specifically,  the  current  research  has  demonstrated  that  the 
metacognitive  processes  hypothesized  to  be  involved  in  pratical  problem  solving  do  exist  and 
can  be  measured.  Furthermore,  this  research  has  demonstrated  that  particular  metacognitive 
processes  play  a  differential  role  in  TK  and  practical  problem  solving.  These  results  support  the 
findings  of  previous  research  investigating  expert/novice  differences  in  performance  which  have 
indicated  that  experts  spend  more  time  on  problem  diagnosis  than  any  other  aspect  of  problem 
solving.  These  results  further  add  to  the  expert/novice  research  by  providing  stable  and  reliable 
measures  of  metacognitive  problem-solving  skills  and  by  indicating  a  relation  between  problem 
representation  and  the  acquisition  of  knowledge  that  is  used  in  problem  solving.  The  current 
research  will  explore  the  symbiotic  relationship  between  knowledge-acquisition  components  and 
metacognitive  processes. 


Potential  Applications 

Our  latest  research  findings  have  specific  implications  for  the  development  of  leadership 
capability.  Current  explicit  leadership  instruction  does  not  provide  a  way  in  which  leaders  can 
translate  leadership  principles  into  actually  solving  practical  leadership  problems.  Because  the 
complex  and  ever-changing  nature  of  job  environments  precludes  teaching  leaders  what  must  be 
done  in  every  problem  situation,  leadership  instruction  must  instead  provide  leaders  with  the 
skills  to  use  experience  as  their  career-long  mentor  and  source  of  feedback.  By  demonstrating 
that  measures  of  the  cognitive  processes  used  in  TK  and  practical  problem  solving  are  stable  and 
reliable,  we  have  made  inroads  to  developing  instructional  materials  that  can  provide  useful 
feedback  to  cadets  or  leaders  participating  in  leadership  training  programs.  Further,  by 
identifying  the  cognitive  processes  most  critical  to  predicting  TK  scores  (i.e.,  those  involved  in 
problem  identification),  we  have  provided  a  critical  target  for  focus  in  leadership  training. 
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Workshop  on  Leadership  at  a  Distance 

Contract  #:  DASW01-00-K-0001  Contract  Dates:  12/18/00  -3/31/03 

Institution:  University  of  Arizona,  PI:  Suzanne  Weisband 

Arizona  State  University  Co  PI:  Leanne  Atwater 

Scientific  Objectives 

Previous  social  science  research  on  issues  related  to  leadership  has  discussed  the  importance 
of  leading  people  remotely  and  the  difficulties  inherent  in  doing  so.  However,  the  empirical 
research  on  leadership  in  geographically  dispersed  work  is  not  well  integrated  nor  well 
understood.  The  rapid  and  pervasive  diffusion  of  new  communication  technologies,  coupled 
with  increased  emphasis  on  distributed  work  units,  is  fundamentally  transforming  the  modem 
work  force,  including  the  military.  With  these  changes  have  come  new  challenges  and 
opportunities  for  the  exercise  of  leadership.  Among  the  challenges  are: 

•  An  increased  amount  of  "remote  work,"  in  which  the  nature  of  work  is  at  a  distance; 

•  Training  with  leaders  at  one  location  and  followers  at  another; 

•  Complex  work  arrangements  where  leaders  and  followers  are  in  multiple  geographic 
locations  at  different  times; 

•  People  are  members  of  multiple  teams  and  report  to  multiple  leaders. 

Approach 

Workshop  organizers  invited  25  top  scholars  for  a  3-day  event  in  October  2001  to  present 
draft  chapters  related  to  leadership  at  a  distance.  Our  goal  was  to  draw  from  multiple  disciplines, 
mainly  in  the  social  sciences  and  leadership  fields.  Each  author  was  responsible  for  presenting 
their  work,  as  well  as  for  reviewing  and  presenting  one  other  chapter.  Each  presentation  (of  the 
paper  and  the  review)  was  then  followed  by  a  10-15  minute  period  of  questions.  A  few  authors 
chose  not  to  attend  at  the  last  minute  due  to  the  events  surrounding  9/1 1  and  apprehension  about 
flying.  These  authors  were  given  the  opportunity  to  present  their  work  via  audio  conference, 
while  local  participants  viewed  the  PowerPoint  slides. 

Progress  and  Application 

Two  deliverables  of  this  workshop  are:  (1)  a  website  where  research  ideas  and  papers  to  be 
published  in  the  book  will  be  shared  by  participants  (abstracts  of  which  will  be  advertised  to  the 
public),  and  (2)  the  publication  of  an  edited  book.  The  website  is  available  at 
http://www.distant-leaders.org.  A  book  proposal  has  been  submitted  to  Lawrence  Erlbaum  and 
Associates  (LEA).  We  are  awaiting  3  chapter  drafts,  all  of  which  are  expected  by  June  15, 2002. 
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Distant  Leadership  Under  Stress 

Contract#:  DASW01-99-K-0003  Contract  Dates:  08/01/1999-07/31/2002 

Institution:  University  of  Maryland,  Baltimore  Pi’s:  Yan  Xiao,  Colin  Mackenzie,  and 

Katherine  Klein 

Scientific  Objectives 

The  overall  objective  of  this  project  is  to  investigate  leadership  effectiveness  when 
leaders  are  distanced  from  the  team.  Since  the  role  of  leadership  in  team  performance  is 
important,  understanding  how  leadership  impacts  team  performance  is  important  to  military  as 
well  as  to  civilian  organizations.  With  the  widespread  use  of  electronic  communication 
technologies,  it  has  become  essential  to  establish  a  theoretical  and  empirical  basis  for  predicting 
how  new  communication  technologies  may  impact  leadership  and  team  performance. 

To  achieve  a  better  understanding  of  distant  leadership  under  stress,  this  research 
program  involves  two  intertwined  efforts.  The  first  is  to  develop  a  conceptual  model  of  the 
interaction  between  task  structure,  stress,  communication  modality,  and  leadership  effectiveness. 
The  second  is  to  conduct  an  empirical  study  of  distant  leadership  using  a  real,  dynamic,  and 
stressful  work  environment  as  a  laboratory.  The  specific  goals  of  this  research  are  as  follows:  (a) 
to  develop  a  matrix  of  leadership  functions  and  situations  in  which  specific  functions  are  needed; 
(b)  to  develop  a  leadership  model  which  prescribes  the  nominal  processes  through  which  a  leader 
controls  and  influences  team  activities,  either  co-located  or  at  a  distance;  (c)  to  develop 
leadership  measures  applicable  to  leadership  in  a  dynamic,  team  environment;  and  (d)  to  conduct 
a  series  of  prospective  studies  in  a  real,  event-driven,  stressful  environment  to  evaluate  the 
impact  of  various  communication  modalities  on  leadership  using  the  measures  developed. 

Approach 

The  theoretic  and  conceptual  efforts  (aims  a-c)  will  be  based  on  existing  literature  on 
team  performance  and  on  the  analysis  of  an  existing  video  library  of  team  performance  collected 
in  a  stressful  environment.  The  cases  from  the  video  library  will  be  reviewed  to  extract  segments 
representative  of  effective  and  ineffective  occurrences  of  leadership.  These  segments  will  be 
used  as  the  basis  for  the  leadership  matrix,  model,  and  measures.  Prospective  empirical  efforts 
(aim  d)  will  utilize  an  already  established  study  environment  in  a  Level-I  trauma  center. 
Comprehensive  audio-video  recordings  will  be  the  primary  data  collection  method.  Two  modes 
of  communication  will  be  randomly  assigned  in  consecutive  cases.  Leadership  effectiveness  will 
be  measured  by  both  team  performance  (speed  and  accuracy)  and  process  parameters  (decision 
processes  and  leadership  control  processes).  These  dependent  measures  will  be  analyzed  in 
relationship  with  several  independent  variables  including  risk,  urgency,  uncertainty,  task 
structure,  and  workload. 


Progress 

Dimensions  of  Leadership  Behavior 

Up  to  this  point,  we  have  selectively  reviewed  existing  literature  on  leadership  functions 
in  team  settings.  One  fruitful  focus  area  has  been  contrasting  the  team  structures  in  trauma 
centers  with  those  structures  used  in  many  other  settings.  Through  our  review  efforts,  it  has 
become  clear  that  teams  in  trauma  centers  have  characteristics  that  are  of  interest  to  the  Army. 
The  review  efforts  also  suggest  significant  gaps  in  the  theoretical  framing  of  leadership 
functions.  Two  manuscripts  are  underway  (one  presented  at  Society  of  Industrial  and 
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Organizational  Psychology  annual  meeting)  to  describe  the  theoretical  progress.  In  particular, 
the  concept  of  shared  leadership,  in  which  different  members  in  a  team  enact  leadership  at 
different  times,  has  been  proposed. 

We  have  conducted  10  interviews  and  70  hours  of  direct  observation  in  the  trauma  center 
to  refine  and  elaborate  our  conceptual  model  guiding  our  study  of  distant  leadership.  In  addition, 
we  gathered  survey  data  from  30  members  of  the  trauma  centers.  Six  leadership  behavior 
dimensions  were  identified  as  frequently  occurring  and  critical  to  team  success.  This  finding 
was  incorporated  into  the  measurement  plan  of  experimental  work. 

Experimentation  on  distance  leadership 

During  the  last  reporting  period,  we  have  successfully  executed  the  most  challenging 
phase  of  the  project:  experimentation  on  leader  location  in  real-life  trauma  patient  resuscitation. 

Overview.  In  order  to  assess  the  potential  impact  of  distant  leadership,  we  outfitted  a 
level-I  trauma  center’s  patient  admitting  area,  where  trauma  patients  are  brought  in  for  initial 
stabilization,  resuscitation,  and  diagnosis.  In  the  distant  leadership  condition,  the  leader  of  a 
trauma  team  (the  attending  trauma  surgeon)  was  told  to  be  at  a  distant  location  where  he  or  she 
could  supervise  the  patient  resuscitation  through  video,  audio,  and  data  link.  To  allow 
comparison,  co-locational  and  distant  leadership  conditions  were  randomized  whenever  logistics 
permitted.  Dependent  measures  included:  (1)  surveys  for  each  session  completed  by  some  of  the 
team  members  before  and  after  the  initial  resuscitation,  (2)  a  debriefing  after  each  session,  (3) 
physiological  measures  of  stress  (saliva  amylase),  (4)  audio-video  recordings  (analysis 
methodology  outlined  below),  and  (5)  patient  status. 

Data  collection  effort  overview.  Surveys,  interviews,  video  and  other  data  were  collected 
from  cases  that  arrived  in  the  TRU  between  10  am  and  5  pm,  Monday  through  Friday  for  two 
blocks  of  dates:  between  March  5  and  April  6  and  between  May  7  and  May  30.  Additionally, 
during  the  designated  hours  in  May,  observational  data  on  non-videotaped  cases  were  collected 
in  which  data  on  team  membership  and  patient  conditions  were  collected  for  cases. 

There  were  59  taped  cases  collected  on  37  separate  days.  Of  the  59  cases,  32  of  them 
involved  local  leadership  condition,  22  involved  remote  (distance)  leadership,  and  5  were 
anomalous,  involving  a  mix  of  distance  and  remote.  An  additional  68  cases  were  observed,  but 
not  taped  on  19  days  (some  overlapping  with  “taped”  days).  Thus,  there  were  a  total  of  127 
cases  either  taped  or  non-taped. 

Of  the  89  participants  involved  in  videotaping,  26  were  staff  (8  surgical  attendants,  5 
anesthesia  attendants,  13  nurses  and  technicians),  and  63  were  in  training  (6  surgical  fellows  and 
chief  residents,  51  residents,  and  6  medical  students). 

Data  analysis.  All  post-session  debriefings  have  been  transcribed.  Initial  analysis  of  the 
transcripts  was  the  basis  for  two  manuscript  submissions.  The  first  was  for  a  workshop  on 
distant  leadership  organized  by  Leanne  Atwater  and  Suzie  Weizband.  The  second  was  for  the 
annual  meeting  of  Human  Factors  &  Ergonomics  Society.  A  central  theme  in  the  debriefing 
materials  is  that  the  teams  adapted  their  structures  in  response  to  two  factors:  (1)  the 
management  of  dual  goals  of  the  team:  training  and  performance,  and  (2)  the  combination  of  task 
conditions  and  team  experience. 

Video  analysis.  A  significant  portion  of  the  video  analysis  has  been  accomplished.  This 
included  a  review  of  videotaped  sessions  by  subject  matter  experts  (who  were  not  part 
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participants  in  the  research).  During  the  review,  each  subject  matter  expert  filled  out  a 
questionnaire  that  provided  his/her  opinions  on  the  leadership  and  team  performance  observed. 

In  addition,  a  measure  of  time  and  accuracy  was  captured  on  a  set  of  pre-defined  landmarks 
during  patient  resuscitation. 

Further  analyses 

A  number  of  data  analysis  activities  will  be  completed  during  the  remainder  of  the 
project.  First,  we  plan  to  carry  out  multivariate  analyses.  Initial  analyses  have  shown  great 
promise  of  some  very  important  research  findings.  Second,  we  plan  to  further  analyze  the 
videotapes  to  extract  segments  highlighting  leadership  situations  and  to  see  how  leadership 
behaviors  changed  accordingly  in  response  to  task  situations,  team  experience,  and  leader 
location.  Third,  we  plan  to  further  code  verbal  and  non-verbal  communications. 

Contributions  to  Basic  Science 

Our  research  has  several  unique  features  as  compared  to  previous  leadership  research. 
First,  we  focus  on  the  performance-based  measurement  of  leadership  effectiveness  in  a  highly 
dynamic  team  environment.  Second,  our  empirical  data  is  from  a  real  environment.  Third,  our 
study  environment  allows  direct  observations  and  audio-video  recordings  of  real  team  activities 
under  stressful  situations  and  the  manipulation  of  communication  modalities  between  leaders  and 
team  members.  Fourth,  the  task  situation  characteristics  (high-risk,  high-tempo)  provide  a  basis 
for  studying  highly  reliable,  rapidly  assembled  and  flexible  teams. 

In  our  initial  analyses,  we  examined  changes  in  team  structures  due  to  the  impact  of 
distant  leadership,  team  experience,  and  task  urgency.  We  proposed  a  set  of  team  structure 
archetypes  as  a  way  to  reflect  the  impact  of  team  experience,  leader  location,  and  task  urgency. 
Previous  studies  on  leadership  and  team  performance  have  not  examined  team  structure  in  this 
manner.  Instead,  research  has  typically  assumed  only  two  team  structures:  1)  undifferentiated 
(such  as  self-managed  teams),  or  2)  a  leader-follower/s  dichotomy. 

Additionally,  our  analyses  thus  far,  based  on  debriefing  materials  and  verbal 
communication,  have  demonstrated  that  leaders  and  teams  adapt  their  structure  both  across 
sessions  and  within  sessions.  We  attribute  our  findings  to  two  potential  causes:  (1)  team 
members  were  differentiated  in  terms  of  experience  and  authority,  and  (2)  teams  were  in  a  real 
environment.  We  believe  that  measuring  team  structures  is  an  effective  way  to  capture  distant 
leadership  and  that  the  findings  associated  with  team  structure  will  have  direct  implications  for 
training  and  team- technology  design. 

Potential  Applications 

We  anticipate  our  efforts  will  fill  a  void  in  the  literature.  In  particular,  our  research  will 
help  us  to  understand  leadership  processes  in  the  context  of  team  performance  and  provide  a 
basis  for  future  training  programs  and  other  interventions  to  enhance  team  performance, 
especially  when  leaders  and  members  are  separated  by  distance. 

Although  data  analysis  is  still  underway,  we  will  highlight  one  potential  impact  of  our 
research  on  team-technology  design.  Due  to  rapid  changes  found  in  some  work  settings,  teams 
may  encounter  dynamic  task  conditions  in  which  they  have  the  dual  goal  of  successful  training 
and  performance.  Our  experimental  results  suggest  that  team  leaders  adapt  how  they 
communicate  with  their  team  members.  This  ability  to  control  and  rapidly  adapt  configurations 
of  communication  channels  (who  talks  to  whom  and  who  can  hear  what)  is  critical.  The  leaders 
in  our  research  showed  preference  for  a  hierarchical  communication  structure,  but  would  alter 
that  structure  when  task  urgency  was  high  or  the  team  was  inexperienced  working  together. 
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Leadership,  Team  Processes,  and  Team  Adaptation: 

The  Development  and  Influence  of  Functional  Leadership  Capabilities  on 

Team  Adaptability  to  Adversity 

Contract  #:  DASW01-98-K-0005  Contract  Dates:  8/1/98-7/31/03 

Institution:  George  Mason  University  PI:  Stephen  J.  Zaccaro 

Co-PI:  Richard  Klimoski 


Scientific  Objectives 

Leadership  and  team  performance  are  critical  elements  of  military  effectiveness.  Yet, 
despite  vast  literatures  on  leadership  and  team  dynamics,  respectively,  there  is  surprisingly  little 
conceptual  research  on  precisely  how  leaders  create  and  direct  team  processes  to  achieve 
collective  success.  In  particular,  there  has  been  little  consideration  in  both  the  teams  and 
leadership  literature  given  to  (a)  the  team  cognitive,  motivational,  and  coordination  processes 
that  specifically  promote  team  adaptation;  (b)  how  leaders  influence  the  quality  of  these 
processes;  (c)  the  personal  qualities  of  the  leader  and  of  team  members  that  promote  team 
adaptation;  and  (d)  leader  and  team  training  principles  that  specifically  foster  team  adaptation. 

The  purposes  of  the  proposed  research  program  are  (a)  to  examine  in  more  detail  the 
influences  leaders  have  on  team  processes  contributing  to  team  adaptation,  (b)  to  examine 
training  and  development  principles  that  contribute  to  the  development  of  adaptive  military 
leaders  and  teams;  and  (c)  to  develop  and  examine  the  psychometric  properties  of  a  leader 
assessment  tool  that  measures  personal  qualities  contributing  to  adaptation.  We  are  meeting 
these  objectives  with  a  series  of  experimental  and  field  studies  in  which  we  examine  leadership 
and  the  motivational,  cognitive,  and  coordination  processes  that  contribute  to  team  adaptation. 
We  also  intend  to  examine  and  validate  integrated  leader  and  team  training  and  development 
guidelines  that  would  foster  these  processes.  Finally,  our  research  efforts  will  also  yield 
assessment  tools  designed  to  measure  the  influence  of  leader  characteristics  on  leadership  and 
team  adaptability. 


Approach 

This  research  effort  is  composed  of  both  experimental  and  field  studies.  The 
experimental  studies  are  intended  to  examine  (a)  leader  attributes  that  promote  leader  and  team 
adaptation,  and  (b)  training  feedback  characteristics  that  foster  team  adaptation.  The  field 
studies  are  designed  to  (a)  validate  an  assessment  battery  constructed  to  measure  multiple 
attributes  related  to  leader  flexibility,  (b)  examine  work  experiences  related  to  the  development 
and  emergence  of  attributes  promoting  leader  flexibility,  and  (c)  test  training  principles  regarding 
the  development  of  leader  flexibility.  We  have  used  the  Army  War  College  as  a  site  for  these 
field  studies.  In  a  related  effort  we  have  cooperated  with  Industrial  College  of  the  Armed  Forces 
to  offer  a  course  in  leader  adaptability. 


Progress 

We  have  completed  four  experimental  studies.  Our  first  study  examined  the  effects  of 
process  versus  outcome  feedback,  and  individual  versus  team-level  feedback  on  team  adaptation. 
We  also  examined  the  role  of  certain  leader  adaptability  attributes  on  team  processes  and 
performance.  In  our  second  study,  we  investigated  other  dimensions  of  training  feedback, 
including  public  versus  private  feedback,  on  team  efficacy  and  team  mental  models,  as  well  as 
team  adaptation.  In  our  third  study,  which  we  recently  completed,  we  examined  the  role  of 
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variability  in  training  performance  scenarios  on  adaptability.  We  are  currently  analyzing  the  data 
from  this  study.  In  our  fourth  study,  that  is  still  ongoing,  we  are  exploring  in  more  detail  the 
nature  of  team  mental  models. 

For  the  field  research,  we  have  developed  conceptual  models  that  describe  the  influences 
of  work  experiences,  formal  instruction,  mentoring  and  coaching  processes,  feedback  systems, 
and  self-development  experiences  on  individual  attributes  related  to  leader  flexibility.  We  have 
also  developed  an  assessment  battery  that  measures  cognitive,  behavioral,  and  dispositional 
attributes  linked  to  leader  flexibility.  We  have  completed  several  validation  studies,  affirming 
the  psychometric  strengths  of  this  battery.  We  also  examined  the  influence  of  these  attributes  on 
the  relationship  between  reported  developmental  work  experiences  and  gains  in  tacit  knowledge 
in  military  officers.  We  have  administered  this  battery  to  several  classes  at  AWC  and  to  a 
sample  of  managers  and  executives  in  several  private  corporations.  We  are  continuing  our 
examination  of  archival  sources  of  work  experience  data  in  order  to  relate  specific  categories  of 
work  experiences  to  the  development  of  key  leader  adaptability  attributes.  Finally,  in 
cooperation  with  the  Center  for  Creative  Leadership,  we  are  developing  criterion  measures  to 
assess  the  relationship  between  leader  adaptability  traits  and  reactions  to  organizational  change 
in  one  of  their  leader  performance  simulations.  This  study  will  allow  us  to  validate  the  proposed 
leader  attributes  as  predictors  of  processes  and  performance. 

In  related  efforts,  we  have  developed  and  pilot-tested  a  proposed  training  curriculum  at 
the  Army  War  College.  This  program  applies  principles  of  adaptability  training  that  were 
derived  from  the  conceptual  models  developed  in  year  one  of  this  effort.  We  have  implemented 
full  versions  of  this  course  at  AWC  in  Spring,  2001  and  at  ICAF  in  Fall,  2002.  These 
applications  of  our  research  did  not  allow  for  a  training  evaluation.  We  are  currently  exploring 
other  venues  for  such  evaluations.  We  are  also  revising  components  of  this  training  for  use  in  a 
adaptive  leader  training  program  being  developed  by  PDRI.  We  will  explore  this  as  a  possible 
venue  for  training  evaluation. 


Contributions  to  Basic  Science 

One  product  of  this  effort  is  a  comprehensive  conceptual  framework  that  describes  a 
career-long  process  of  leader  development.  This  framework  builds  on  previous  research  in  the 
literature  and  incorporates  multiple  sources  of  developmental  experiences  (i.e.,  formal  training, 
self-development,  work  assignments,  mentoring,  coaching,  feedback).  Accordingly,  it  is 
particularly  appropriate  for  the  U.S.  Army.  Also,  the  data  from  the  planned  laboratory  and  field 
studies  should  validate  training  principles  regarding  the  efficacy  of  particular  kinds  of 
interventions  designed  to  grow  adaptability  skills.  These  data  should  also  provide  information 
regarding  the  development  and  influences  of  leader  and  team  attributes  that  promote  adaptability. 
Finally,  we  have  a  growing  database  that  provides  validity  for  proposed  relationships  between 
certain  key  leader  attributes  and  adaptability. 

Potential  Applications 

The  applications  of  this  research  for  the  Army  are  expected  to  be  in  the  areas  of  training 
assessment  and  curriculum.  If  validated,  the  assessment  battery  measuring  leader  flexibility  can 
serve  as  a  very  effective  training  tool  in  multiple  Army  training  settings.  Further,  the  field 
studies  are  expected  to  yield  specific  guidelines  and  curriculum  tools  that  target  the  development 
of  leader  flexibility.  These  guidelines  and  tools  are  expected  to  be  constructed  in  accordance 
with  the  different  developmental  needs  that  emerge  at  various  points  in  an  officer’s  career.  Also, 
the  adaptability-training  program  being  implemented  at  AWC  should  have  direct  applicability  to 
the  development  of  adaptive  senior  leaders. 
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RACO  RESEARCH  OBJECTIVE  #3: 

Provide  fundamental  knowledge  for  identifying  and  measuring 
the  attributes  and  skills  that  are  critical  to  soldier  recruiting, 
performance,  and  retention  in  the  transforming  Army. 


Research  in  this  section  is  directed  toward  identifying  and  measuring 
the  aptitudes  and  skills  that  are  unique  to  the  human  performance 
requirements  of  the  Objective  Force.  Exploring  the  sociological  and 
psychological  factors  that  could  influence  recruitment,  retention,  and 
Army  performance  are  important  aspects  of  this  research  objective. 
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International  Military  Education  and  Training:  A  Sociological  Analyses 

Contract  #:  DASW01-98-K-0003  Contract  Dates:  6/1/1998-12/31/2001 

Institution:  Northwestern  University  PI:  Charles  Moskos 

Scientific  Objectives 

The  research  bridges  methodological  and  theoretical  issues  found  both  in  social  science 
dealing  with  military  organization  and  those  dealing  more  generally  with  cross-cultural 
cooperation  in  international  organizations.  The  research  advances  social  science  understanding 
of  intercultural  and  interpersonal  relations  within  large-scale  organizations.  One  specific  goal  is 
to  identify  factors  that  help  or  hinder  effectiveness  within  multinational  military  organizations. 

As  with  prior  ARI-sponsored  projects,  the  research  agenda  is  also  subject  to  direction  from 
senior  officials  in  the  military  and  U.S.  government.  The  research  goals  are  explicitly  dynamic 
in  order  to  incorporate  developments  during  the  period  of  the  research. 

Approach 

Since  1950,  some  500,000  international  military  officers  and  senior  enlisted  personnel 
have  received  professional  military  education  through  the  International  Education  and  Training 
(IMET)  program  of  the  United  States.  In  the  Post-Cold  War  era,  IMET  plays  an  even  more 
important  role  with  regard  to  multinational  operations  that  encompass  nontraditional  missions 
(e.g.  peacekeeping,  humanitarian,  and  nation  building)  as  well  as  conventional  military  concerns. 
This  research  has  attained  new  significance  since  the  events  of  11  September  2001  and  the 
ensuing  war  against  terrorism. 

The  approach  is  threefold:  (1)  research  on  IMET  participants  in  the  United  States,  (2) 
research  on  IMET  graduates  in  their  home  countries,  and  (3)  research  on  comparable  programs 
in  other  advanced  Western  democracies.  The  methodology  is  primarily  in-depth  interviews  with 
students,  graduates,  and  staff  members  of  professional  military  education  (PME)  programs. 

Progress 

The  first  phase  of  the  IMET  study  is  completed  and  was  submitted  as  a  Technical  Report 
to  ARI  in  September  2001.  Interviews  were  conducted  in  the  war  and  staff  colleges  in  each  of 
the  services:  Army,  Navy,  Air  Force,  and  Marine  Corps. 

The  second  phase  of  the  IMET  research  is  also  completed  and  involved:  (a)  interviewing 
IMET  alumni  in  their  home  countries  (10  interviews)  and  (b)  comparisons  of  the  American 
IMET  program  with  equivalent  PME  programs  in  the  United  Kingdom  and  Netherlands. 
Additionally,  the  PI  was  personally  invited  by  the  SACEUR  to  conduct  interviews  (April  2001) 
at  NATO  headquarters  on  issues  of  multinational  cooperation.  The  cooperation  extended  the  PI 
by  the  host  institutions  was  extraordinary.  In  addition  to  presentations  to  senior  military  officers, 
briefings  were  requested  by  senior  officials  of  the  National  Defense  University  and  the  Army 
War  College,  as  well  as  the  U.S.  State  Department. 

Contributions  to  Basic  Science 

The  data  along  with  the  concepts  introduced  in  this  research  will  inform  social  scientists 
doing  basic  research  in  several  areas.  One  is  the  role  of  multicultural  communication  with 
formal  organizations.  A  second  is  to  document  the  relative  weight  of  cultural,  organizational, 
and  economic  variables  in  fostering  or  reducing  levels  of  cooperation  among  employees  in 
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civilian  organizations  as  well  as  among  officers  in  military  organizations.  Perhaps,  most 
significantly,  it  will  help  identify  distinctions  between  personality  factors  and  sociological 
variables  in  multinational  organizations. 

Potential  Applications 

Overall,  IMET  is  a  most  cost-effective  way  of  enhancing  international  military 
cooperation.  One  notable  finding  was  that  many  military  officers  often  feel  closer  other  officers 
of  different  nationalities  in  multinational  organizations  than  to  fellow-national  civilians  in  the 
same  organization.  Certain  problem  areas  were  analyzed  with  attendant  recommendations  for 
improvement.  These  include:  (1)  make  medical  insurance  available  to  the  family  members  of  all 
foreign  officers;  (2)  decrease  the  class  material  unavailable  to  international  officers  because  of 
security  classification;  (3)  add  more  curriculum  content  on  multinational  operations;  (4) 
encourage  more  cross-national  activities,  e.g.  informal  dinners,  excursions;  (5)  encourage 
American  officers  to  become  more  familiar  with  the  home  country  of  international  officers  with 
whom  they  regularly  work;  and  (6)  extend  more  effort  to  incorporate  the  spouses  of  international 
officers  into  the  American  social  scene. 


40 


Social  Structures  Affecting  Army  Performance 

Contract  #:  DASW  0100  K  0016  Contract  Dates:  8/17/2000-  8/16/2004 

Institution:  University  of  Maryland  at  Pis:  David  R.  Segal  and  Mady  Wechsler 

College  Park  Segal 


Scientific  Objectives 

This  program  of  research  is  focused  on  the  identification  and  analysis  of  changes  in  the 
international  environment,  American  society,  the  U.S.  Army,  and  the  American  soldier  that  are 
consequential  for  the  fulfillment  of  Army  missions.  The  research  program  is  organized  into  four 
related  areas:  (a)  research  on  American  populations,  (b)  research  on  soldiers  and  their  families, 
(c)  diversity  in  the  military,  and  (d)  comparative  military  institutions. 


Approach 

The  research  approaches,  both  qualitative  and  quantitative,  vary  among  our  four  program 
areas,  as  follows: 

Research  on  American  Populations 

Most  of  our  research  in  this  area  has  focused  on  the  youth  population  from  which  the 
Army’s  recruits  are  drawn.  In  particular,  collaborating  with  colleagues  at  the  University  of 
Michigan,  we  utilize  the  Monitoring  the  Future  (MtF)  database.  This  program  of  research 
surveys  about  16,000  high  school  seniors  each  spring  semester  and  has  been  doing  so  since  1975 
(the  second  year  of  the  current  volunteer  force).  Sub-samples  of  seniors  are  then  surveyed  semi¬ 
annually  for  several  years  after  high  school  graduation.  Among  the  areas  covered  in  these 
surveys  are  military-related  attitudes  and  behaviors. 

Research  on  Soldiers  and  their  Families 

In  this  area,  we  conduct  surveys  of  soldiers  in  selected  units  and  surveys  of  military 
wives.  We  also  conduct  interviews  with  soldiers  and  spouses,  conduct  observations  of  units  in 
the  field,  and  draw  upon  documentary  materials  in  a  variety  of  archives. 

Diversity  in  the  Military 

Our  concern  here  is  how  individuals  adapt  to  the  military  and  its  missions  and  how  this 
adaptation  is  affected  by  race,  ethnicity,  gender,  and  sexual  orientation.  We  draw  upon  our  own 
surveys  and  observations,  secondary  analysis  of  other  surveys,  and  field  observations. 

Comparative  Military  Institutions 

Our  goal  here  is  to  draw  upon  the  experiences  of  foreign  military  forces  to  identify  cross¬ 
national  similarities  and  differences  as  well  as  to  learn  from  the  experiences  of  others.  Our 
approaches  have  been  (1)  to  encourage  cross-national  groups  of  behavioral  scientists  to  engage 
in  parallel  research  efforts  in  their  own  countries  and  to  pool  the  knowledge  acquired  and  (2)  to 
conduct  surveys  within  the  armed  forces  of  other  nations. 


41 


Progress 

Seven  research  reports  have  been  published  during  this  first  contract  year  including  four 
journal  articles  and  three  book  chapters.  Copies  of  these  have  been  provided  to  ARI.  An 
additional  five  publications  have  been  submitted  for  publication  review.  Seven  additional  papers 
were  presented  at  professional  conferences.  In  the  following  section,  we  provide  an  overview  of 
the  research  findings  we  have  published. 

Research  on  American  Populations 

Regarding  American  youth,  the  major  research  published  this  year  dealt  with  (1)  sex 
differences  in  high  school  seniors’  propensity  to  enlist  in  the  military,  (2)  differences  between 
military  preferences  and  expectations,  (3)  sex  differences  in  the  relationship  between  propensity 
and  actual  service,  (4)  effects  of  background  and  educational  characteristics  on  propensity  and 
enlistment,  and  (5)  the  post-high  school  activities  of  men  and  women  who  do  not  serve.  Using 
data  from  the  MtF  surveys,  we  found  that  young  women’s  propensity  to  serve  is  lower  than 
men’s,  but  that  more  women  desire  than  expect  to  serve.  The  relationship  between  propensity 
and  service  is  weaker  for  women  than  men,  and  these  relationships  have  not  increased  over  time. 
Background  characteristics,  educational  achievement,  and  educational  plans  are  less  predictive  of 
women’s  propensity  and  enlistment  than  men’s,  with  the  exception  of  greater  race  and  ethnicity 
effects  among  women.  Having  children  has  a  small  negative  effect  for  low  propensity  men’s 
enlistment.  Lastly,  more  women  desire  and  expect  to  serve  than  the  military  is  enlisting. 

Research  on  Soldiers  and  Their  Families 

Our  research  on  military  personnel  has  addressed  issues  of  downsizing  and  deployment 
tempo.  In  one  of  our  survey  research  efforts,  we  analyzed  the  impact  of  multiple  deployments  on 
peacekeeping  attitudes,  personal  morale,  and  reenlistment  intentions  of  American  soldiers.  In 
terms  of  peacekeeping  attitudes,  the  greatest  effects  were  on  judgments  of  the  appropriateness  of 
alternative  personnel  to  perform  peacekeeping  operations.  With  respect  to  morale  and 
reenlistment  intentions,  we  found  support  for  the  negative  impact  of  multiple  deployments  on 
personal  morale.  However,  we  found  no  statistical  support  for  the  relationship  between  number 
of  deployments  and  reenlistment  intentions. 

A  more  socio-historical  view  of  military  service  in  the  United  States  sought  to  clarify  the 
origins  of  citizen  service  in  the  United  States.  It  departed  from  the  premise  that  military  service 
compelled  by  the  state  and  performed  as  an  obligation  of  citizenship  may  be  understood  as 
constitutive  of  citizen  service.  Based  on  this  analytic  distinction,  the  article  developed  one 
central  argument.  Citizen  service  was  first  realized  in  the  United  States  during  the  American 
Revolution,  but  not  as  might  be  expected  in  the  compulsory  militia  dating  from  the  colonial 
period.  Rather,  citizen  service  was  realized  first  in  the  form  of  the  federally  mandated 
conscription  of  American  national  citizens  into  the  Continental  Army,  a  peculiarly  Revolutionary 
contribution  to  the  War  of  Independence.  Citizen  service’s  Revolutionary  birth  in  national 
conscription,  then,  helps  to  recast  the  very  roots  of  the  American  military  tradition. 

Research  on  Diversity  in  the  Military 

Our  research  on  gender  and  propensity  to  serve  in  the  military  (also  discussed  above  as 
part  of  our  research  program  on  American  youth)  falls  into  the  diversity  area  as  well.  One  report 
based  on  this  research  discusses  ways  in  which  the  military  creates,  sustains,  and  reflects  gender 
and  sexuality  norms,  stereotypes,  and  stratification.  In  particular,  the  relationships  between  the 
military,  gender,  and  sexuality  can  be  examined  at  multiple  levels.  At  the  societal  level,  the 
paper  discusses  ways  in  which  the  military  affects  the  social  constructions  of  gender  and 
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sexuality,  as  well  as  how  society’s  gender  structure  affects  women’s  military  participation.  At 
the  organizational  level,  it  describes  the  current  legal  and  demographic  status  of  women  and 
homosexuals  in  the  U.S.  military.  At  the  individual/interpersonal  level,  it  describes  how  gender 
and  sexuality  are  constructed  and  reconstructed  through  the  daily  interactions  of  military 
members.  Included  are  summaries  of  the  participation  of  women  and  homosexuals  in  the  armed 
forces  and  the  policy  conditions  of  their  service. 

Research  on  Comparative  Military  Institutions 

In  regards  to  our  research  program  on  comparative  military  institutions,  part  of  our 
research  focused  on  the  Russian  Army.  The  Russian  Army,  like  many  modem  military  forces, 
underwent  a  major  downsizing  during  the  mid-1990s.  This  project  intersects  with  our  research 
on  soldiers  and  their  families.  We  conducted  an  analysis  of  quality  of  life  and  mental  health 
survey  data  of  Russian  army  officers  and  their  wives  in  four  conditions:  (1)  those  who  survived 
downsizing;  (2)  those  still  in  the  army  but  expecting  to  leave;  (3)  those  who  left  and  found 
civilian  employment;  and  (4)  those  who  left  and  had  not  yet  found  civilian  jobs. 

The  groups  reporting  the  highest  quality  of  life  were  not  the  survivors  of  downsizing,  as 
has  been  found  in  most  downsizing  research,  but  rather  those  who  had  left  the  army  and  found 
civilian  jobs.  The  same  pattern  held  for  the  officers’  wives.  With  regard  to  mental  health,  the 
group  reflecting  the  highest  levels  of  depression,  anxiety,  and  hostility  were  those  soldiers  who 
were  still  in  the  army  but  were  expecting  to  leave,  while  the  group  reporting  the  least  distress 
were  employed  veterans.  Again,  the  same  pattern  held  for  the  officers’  wives.  Thus,  leaving  the 
Russian  Army  through  downsizing  produced  no  more  distress  than  surviving  downsizing.  This 
fact  may  make  Russia’s  proposed  transition  from  a  conscription-based  army  to  a  professional 
force  difficult  to  achieve. 

A  more  comparative  research  effort  in  this  area  presents  a  historical  overview  of  the  field 
of  military  sociology  as  a  context  for  discussing  the  kinds  of  structural  changes  that  have  taken 
place  in  military  organization  since  World  War  II.  It  describes  the  emergence  of  a  “late  Cold 
War”  military  structure  after  the  Vietnam  War  and  of  a  “Post-Cold  War”  military  after  the 
collapse  of  the  Soviet  Union.  Characteristics  of  this  post-Cold  War  structure  are  discussed. 
Reductions  in  defense  expenditures  and  in  military  force  size  in  the  nations  of  Western  Europe 
and  Latin  American  are  presented. 

Another  comparative  effort  dealt  with  the  conceptualization  of  change  in  military 
organizations.  Social  scientists  concerned  with  the  processes  of  change  within  military 
organizations  have  begun  to  appropriate  the  language  of  postmodernism  in  order  to  describe  a 
number  of  transformations  observed  in  military  forces  cross-nationally  since  the  end  of  the  Cold 
War.  Although  the  thesis  of  the  “postmodern  military”  may  have  a  novel  appeal  to  scholars 
across  a  broad  range  of  disciplines,  we  argue  that  while  substantial  changes  are  indeed  occurring 
in  the  armed  forces  of  Western  countries,  there  is  at  present  little  evidence  to  suggest  that  these 
changes  are  postmodern  in  nature.  On  the  contrary,  most  of  the  developments  in  contemporary 
armed  forces  being  described  as  such  are  actually  modem  in  character — emerging  as  the  result  of 
rational,  purposeful  adaptation  to  environmental  contingencies.  We  argue  that  the  intellectual 
discourse  of  postmodernism  is  useful  for  military  sociology  only  to  the  extent  that  scholars  are 
able  to  apply  discrete  strands  of  postmodern  social  theory  to  predict  or  explain  processes  of 
change.  We  examine  the  utility  of  three  such  stands:  post-industrialism,  post-Fordism,  and 
globalization.  Finally,  alternate  criteria,  heretofore  marginalized  in  the  discussion  of  the 
postmodern  military,  are  suggested  as  benchmarks  to  determine  the  degree  to  which  the  armed 
forces  have  indeed  become  postmodern. 
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Contributions  to  Basic  Science 

Much  of  our  work  feeds  directly  into  ongoing  paradigm  shifts  in  the  behavioral  and  social 
sciences.  The  nature  of  the  human  life  course  has  been  a  major  paradigm.  In  particular,  our 
work  on  the  antecedents  of  military  service  has  helped  shape  how  decisions  made  by  young 
people  about  their  post-high  school  trajectories  are  treated  from  this  theoretical  perspective. 
Similarly,  our  work  on  the  post  service  experiences  of  those  who  have  worn  the  uniform  extends 
our  understanding  of  the  other  extreme  of  life  course  trajectories.  Complementing  this,  our  work 
on  military  families  has  fed  directly  into  a  paradigm  shift  in  the  study  of  labor  market  dynamics 
that  increasingly  focuses  on  social  structural  determinants  rather  than  characteristics  of 
individuals. 

Methodological  advances  have  accompanied  these  conceptual  contributions. 

Specifically,  we  have  expanded  the  utilization  of  hierarchical  linear  modeling  as  a  tool  to 
measure  the  effects  of  social  structures  (such  as  military  bases  or  military  units).  This  allows  us 
to  obtain  more  reliable  estimates  of  these  effects  than  can  be  derived  from  the  more  traditional 
Ordinary  Least  Squares  models  or  their  single-level  variants. 

Postmodern  models  postulate  revolutionary  breaks  from  the  past  and  reject  much  of  the 
canon  that  has  been  accepted  in  theorizing  about  the  military.  Our  work  on  organizational 
change  has  been  extremely  influential  in  shaping  the  debate  among  behavioral  scientists  who 
analyze  the  military  on  the  degree  to  which  these  postmodern  models  can  be  usefully  adopted  as 
a  strategy  for  “out  of  the  box”  thinking. 

Potential  Applications 

Our  research  has  a  number  of  obvious  potential  applications.  First,  the  research  on 
American  youth  attitudes  toward  military  service  and  the  military  as  an  organization  has 
implications  for  recruitment  strategies.  Second,  our  labor  market  dynamics  research  highlights  a 
problem  area  that  policy-makers  concerned  with  the  quality  of  military  life,  as  a  factor  in  the 
retention  of  military  personnel,  must  attend  to.  Third,  our  research  on  the  attitudes  and  behavior 
of  military  personnel  can  help  inform  Army  doctrine  on  the  kinds  of  soldiers  that  should  be  used 
in  stability  operations  and  special  training  they  may  require.  Finally,  our  cross-national  research 
allows  us  to  identify  human  resource  management  strategies  that  have  been  implemented  in  other 
nations  that  might  be  useful  in  the  U.S. 
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Strategic  Recruitment  and  Retention: 

Preference  Structure,  Expectations,  and  Choice 

Contract  #:  DASW01-00-K-0011  Contract  Date:  8/1/00  to  7/31/02 

Institution:  California  State  University,  PI:  Mary  Kay  Stevenson 

Hayward 


Scientific  Objectives 

The  purpose  of  this  research  program  is  to  develop  a  general  model  of  how  individuals 
evaluate  job  opportunities  that  involve  both  current  and  future  benefits  when  choosing  to  accept 
a  position  and  stay  in  a  position.  The  studies  that  are  described  in  this  proposal  use  temporal 
discounting  theory  and  theories  of  decision  making  to  assess  the  relevance  of  laboratory  findings 
to  career  decisions.  Research  in  temporal  discounting  has  indicated  that  future  incentives  can 
have  less  impact  than  current  outcomes  in  some  contexts.  However,  the  data  for  this  research 
has  been  primarily  generated  by  asking  subjects  about  fictitious  or  relatively  limited  outcomes. 
The  approaches  are  characteristic  of  studies  that  analyze  groups  of  subjects.  Image  theory  has 
proposed  a  mechanism  for  structuring  the  screening  of  decision  options  and  monitoring  the 
relationship  of  an  individual’s  expectations  with  their  experience.  The  current  theory  uses  Policy 
Modeling  to  create  an  analysis  of  those  expectations  that  focus  on  individual  differences. 
Recruitment  and  retention  strategies  will  be  developed  based  on  the  derived  preference  structures 
and  decision  theory.  By  modeling  the  impact  of  uncertainty  on  the  recruits’  expectations  and  the 
stability  of  the  preference  structures  over  time,  we  can  predict  when  individuals  are  most  likely 
to  leave  a  position  and  the  actions  most  likely  to  circumvent  that  loss. 

Approach 

Last  year  we  began  by  surveying  three  samples:  (1)  a  sample  of  military  personnel  that 
had  just  entered  military  service,  (2)  a  sample  of  those  that  had  completed  at  least  one  year,  and 
(3)  a  sample  of  captains  and  lieutenants.  Within  each  of  these  groups,  we  obtained  responses 
from  soldiers  who  were  ready  to  leave  the  military  and  soldiers  who  were  committed  to  staying 
in  the  Army.  Although  there  were  individuals  differences  in  the  issues  that  affected  their 
decisions  to  stay  or  leave,  it  was  clear  that  issues  related  to  job  satisfaction  were  shared  by  most 
of  the  respondents.  We  were  also  able  to  obtain  some  suggestions  from  the  respondents 
indicating  what  they  felt  would  make  a  difference  in  their  view  of  the  military.  We  are  currently 
preparing  to  replicate  that  sampling  procedure  with  updated  questions  based  on  the  data  collected 
last  year.  Questions  were  revised  to  focus  on  the  most  salient  problems  found. 

A  model  that  incorporated  two  components  to  predicting  satifaction  with  military  life  was 
tested.  This  model  allows  for  individual  differences  in  dissatisfaction  while  predicting 
individuals’  intentions  to  stay  in  the  military.  The  prediction  is  based  on  the  discrepancies 
between  what  they  expected  when  entering  the  Army,  what  they  experienced,  and  how  they 
imagine  the  demands  of  a  comparable  civilian  position.  Initial  study  results  with  a  relatively 
small  sample  looked  promising  and  we  are  looking  forward  to  replicating  the  tests  using  a  larger 
sample. 

We  were  also  able  to  determine,  using  last  year’s  samples,  that  the  temporal  discounting 
data  for  salary  increases  over  2  to  4  years  replicated  results  obtained  with  civilian  participants 
and  those  reported  in  the  literature.  We  are  currently  working  with  this  data  to  develop  a 
detained  accounting  of  the  preference  structures  for  raises. 
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Also,  we  are  currently  preparing  to  replicate  these  findings  using  a  civilian  organization. 
Although  we  have  been  collecting  some  data  from  students,  most  of  the  students  samples  have 
expressed  a  strong  preferece  not  to  enter  the  military  service.  By  changing  our  focus  to  adults 
who  have  positions  in  a  large  civilian  organization,  we  can  compare  their  expectations  and  job 
related  issues  to  the  informatin  we  obtained  in  the  military  setting.  Our  analyses  will  focus  on 
individual  differences  in  preferences,  concerns,  and  expectations  that  determine  the  outcome  of 
decisions  to  seek  jobs  or  change  jobs. 


Progress 

Since  August  1,  2000,  the  following  tasks  have  been  completed: 

1.  I  trained  two  students  (Kristen  Christy  and  Lisa  Beasley)  to  assist  in  the  data  collection  and 
preliminary  analyses. 

2.  We  began  collecting  data  with  civilians  on  their  attitudes  and  expectations  about  civilian  and 
military  careers  in  October  2000.  This  pilot  data  was  used  to  modify  the  surveys. 

3.  We  completed  preliminary  descriptive  analyses  of  the  pilot  data.  We  are  adding  to  the 
civilian  sample  size  and  diversity  this  quarter  using  the  revised  surveys. 

4.  Kristen  and  I  collected  data  at  Fort  Bliss  on  Feb.  5,  2001  and  at  Fort  Hood  on  April  2, 2001. 
We  completed  the  analyses.  I  reported  some  of  the  results  we  obtained  to  officers  at  ARI  in 
August  of  2001.  I  am  currently  finishing  a  techical  report  on  these  results. 

5.  I  trained  Loretta  Hill  to  continue  in  Kristen’s  place.  Loretta  is  collecing  data  at  Los  Positas 
College  and  the  Business  School  to  increase  the  male  respondents  in  our  sample  before  June. 

6.  Loretta  and  I  will  also  be  collecting  data  at  Fort  Bliss  at  the  beginning  of  March  and  at  Fort 
Carson  in  May. 

7.  Lisa  has  programmed  the  surveys  on  a  Web  Site  and  will  be  collecting  data  from 
organizations  in  about  1  month. 

Contributions  To  Basic  Science 

These  studies  will  involve  the  development  of  new  modeling  and  measurement 
techniques.  By  using  Image  Theory  and  Policy  Modeling  it  will  provide  a  test  of  the  theories 
and  extend  information  about  decision  making  in  the  context  of  career  decisions.  Finally,  we 
hope  to  generate  survey  instruments  that  can  be  used  to  measure  peoples'  attitudes  toward  work 
in  a  way  that  provides  information  on  strategies  to  recruit  and  retain  valued  personnel.  We  also 
hope  to  include  practical  suggestions  for  changes  that  may  improve  retention. 

Potential  Applications 

The  application  of  this  work  is  relevant  to  any  organization  that  recruits  a  work  force, 
wishes  to  retain  valued  employees,  and  wants  to  anticipate  when  these  employees  are  likely  to 
look  for  alternative  positions.  The  results  from  this  research  will  identify  motivational  resources 
to  assist  in  attracting  and  retaining  talented  personnel.  In  general,  people  are  motivated  when  the 
compensation  package  meets  their  expectations  (Vroom,  1964)  and  when  they  perceive  that  they 
are  being  treated  equitably  (Adams,  1965).  By  monitoring  our  participants’  experiences  and 
interpreting  these  experiences  according  to  their  expectations  about  the  job  and  competing  jobs, 
we  can  use  this  feedback  to  predict  dissatisfaction  and  stimulate  change. 
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Personnel  Turnover  and  Team  Performance 


Contract  #:  DASW01-00-K-0018 
Institutions:  University  of  Pittsburgh 

and  Carnegie  Mellon  University 


Contract  Dates:  9/1/00-11/30/03 
Pis:  John  Levine,  Linda  Argote, 
Kathleen  Carley,  &  Richard  Moreland 


Scientific  Objectives 

One  of  the  most  daunting  challenges  for  work  teams  is  personnel  turnover.  Personnel 
turnover  in  work  groups  is  defined  as  the  entry  of  new  members  and/or  the  exit  of  old  members. 
This  challenge  is  particularly  critical  in  organizations,  such  as  the  Army,  that  must  cope  with 
downsizing  and  frequent  personnel  change.  Turnover  represents  a  change  in  team  composition 
that  can  have  profound  consequences  for  team  performance.  It  alters  the  distribution  of 
knowledge  within  the  team  as  well  as  relations  among  team  members.  Thus,  how  to  deal 
productively  with  turnover  is  a  critical  question  for  both  theoretical  and  practical  reasons. 

We  propose  that  the  impact  of  turnover  is  mediated  by  changes  in  the  quantity,  quality, 
and  distribution  of  task-relevant  knowledge  within  a  team.  Such  knowledge,  which  can  be 
conceptualized  in  several  ways  (Klimoski  &  Mohammed,  1994),  can  have  an  important  impact 
on  team  performance  (Argote,  Gruenfeld,  &  Naquin,  2001;  Cannon-Bowers  &  Salas,  1998). 
Personnel  turnover  has  negative  consequences  to  the  extent  that  it  reduces  the  total  knowledge 
base  of  the  team  and/or  members’  ability  to  exchange  knowledge  efficiently.  In  contrast, 
turnover  has  positive  consequences  to  the  extent  that  it  augments  the  team  s  knowledge  base 
and/or  members’  ability  to  send  and  receive  this  knowledge. 

In  addition,  we  assume  that  the  knowledge  changes  associated  with  turnover  both 
influence  and  are  influenced  by  social  interactions  among  team  members.  When  old  members 
leave  a  team,  typically  depleting  the  team’s  knowledge  base  and  disrupting  well-learned  patterns 
of  knowledge  distribution,  remaining  members  must  renegotiate  their  responsibilities  for  storing 
and  sharing  information.  When  new  members  enter  a  team,  they  must  acquire  specific 
knowledge  relevant  to  their  role  as  well  as  knowledge  about  others’  competencies  and 
responsibilities  (Levine  &  Moreland,  1999).  This  socialization  process  can  be  quite  challenging 
if  newcomers  must  learn  large  amounts  of  information  and  old-timers  do  not  transmit  this 
information  efficiently.  In  addition,  newcomers  may  introduce  useful  changes  in  how  the  team 
performs  its  task  (Levine,  Choi,  &  Moreland,  in  press;  Levine,  Moreland,  &  Choi,  2001).  This  is 
particularly  likely  when  the  team  is  facing  new  challenges  and  newcomers  possess  task-relevant 
knowledge. 

Approach 

The  proposed  research  utilizes  two  team  tasks  (decision  making  and  production)  and 
employs  two  methodologies  (laboratory  experimentation  and  computer  simulation).  Three 
related  lines  of  work  are  being  conducted.  First,  studies  using  the  experimental  version  of  the 
decision-making  task  investigate  how  newcomers  help  teams  adapt  to  challenges.  In  this 
research,  team  members  use  a  computer  network  to  obtain  and  share  rapidly  changing 
information  in  an  airplane  surveillance  scenario.  Second,  studies  using  the  experimental  version 
of  the  production  task  investigate  how  teams  limit  the  negative  impact  of  newcomers  on 
transactive  memory  systems  (cf.  Moreland,  1999;  Moreland,  Argote,  &  Knshnan,  1998).  In  this 
research,  team  members  work  together  on  a  challenging  construction  project  (i.e.,  building 
transistor  radios).  Third,  our  simulation  work  seeks  to  create  a  unified  approach  to  turnover  that 
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integrates  work  in  the  decision-making  and  production  domains  and  extends  our  analysis  to 
larger  groups  and  longer  time  periods  than  can  be  readily  investigated  in  the  lab.  The  computer 
simulation  studies  are  virtual  experiments  designed  to  match  and  extend  both  sets  of  laboratory 
experiments  by  adapting  powerful  computer  programs  (ORGAHEAD  and  CONSTRUCT)  that 
have  proven  useful  in  modeling  the  performance  and  dynamic  behavior  of  groups  and 
organizations  (e.g.,  Carley,  1991;  Carley  &  Svoboda,  1996). 

Progress 

During  2001,  we  made  substantial  progress  on  all  three  components  of  our  research.  An 
initial  experiment  using  the  decision-making  task  investigated  how  a  team’s  level  of  choice  in 
selecting  a  task  strategy  and  subsequent  performance  using  that  strategy  affected  the  team’s 
receptivity  to  a  newcomer-initiated  innovation.  The  second  study  in  this  series,  currently 
underway,  examines  how  receptivity  to  newcomer  innovation  is  influenced  by  the  newcomer’s 
confidence  in  his/her  suggested  change  and  the  team’s  perception  of  the  utility  of  its  current 
strategy. 

Our  initial  experiment  using  the  production  task  investigated  how  expected  and  actual 
turnover  affected  team  members’  transactive  memory  and  performance.  The  second  study  in  this 
series,  currently  in  progress,  examines  how  information  about  newcomers’  capabilities  can 
mitigate  the  disruptive  effects  of  turnover. 

In  our  computer  simulation  work,  we  have  developed  computational  models  for  simulating 
group  performance  in  the  decision-making  and  production  task  domains.  We  are  running  virtual 
experiments  to  (a)  validate  the  models  against  experimental  findings,  (b)  examine  the  impact  of 
databases  as  substitutes  for  transactive  memory  systems,  and  (c)  examine  the  effects  of  turnover 
in  larger  scale  systems.  Our  initial  simulation  work  involved  creating  ORGMEM  (an  extension 
of  ORGAHEAD)  to  match  the  transactive  memory  experiments  and  conducting  a  series  of 
virtual  experiments  using  this  simulation.  We  have  subsequently  developed  an  agentized  version 
of  CONSTRUCT,  which  has  ORGMEM  embedded  in  it,  and  used  that  to  replicate  and  extend 
the  transactive  memory  experiments.  We  have  also  extended  the  ORGAHEAD  multi-agent 
network  model  and  used  that  to  explore  how  newcomers  impact  performance.  We  are  in  the 
process  of  replicating  lab  experiments  in  this  area. 

Contributions  to  Basic  Science 

Only  a  handful  of  studies  have  examined  personnel  turnover  in  teams.  These  studies 
suggest  that  turnover  typically  disrupts  team  performance,  but  is  more  deleterious  under  some 
conditions  than  others  (e.g.,  Argote,  Insko,  Yovetich,  &  Romero,  1995).  Because  these  studies 
vary  widely  on  conceptual  as  well  as  methodological  grounds,  their  results  are  difficult  to 
integrate  in  a  theoretically  satisfying  manner.  Therefore,  a  major  goal  of  the  present  research 
program  is  to  provide  a  stronger  theoretical  framework  for  understanding  the  consequences  of 
turnover  (positive  as  well  as  negative)  for  work  teams.  We  believe  our  use  of  simulation  in 
conjunction  with  experimental  studies  will  be  helpful  in  achieving  this  goal.  The  simulation 
studies  will  allow  us  to  examine  a  wider  range  of  parameter  values  than  is  possible  in  the 
laboratory,  whereas  the  experimental  studies  will  help  us  refine  and  elaborate  the  simulation 
models.  In  addition  to  clarifying  the  effects  of  turnover  in  teams,  our  research  program  will 
generate  valuable  information  on  how  social  and  organizational  scientists  can  work  together  in 
linking  laboratory  experimentation  and  computational  analysis. 
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Potential  Applications 

Collaborative  work  is  an  increasingly  important  aspect  of  organizational  life,  and  many 
organizations  assign  their  most  critical  tasks  to  work  teams  (Ilgen,  1999;  Sundstrom,  1999). 
Unfortunately,  however,  teams  do  not  always  provide  the  benefits  they  promise  (Hackman, 

1990).  Failure  to  adapt  to  personnel  turnover  is  one  reason  teams’  actual  productivity  falls  below 
their  potential  productivity.  Turnover  can  be  harmful,  for  example,  when  a  team  loses  a 
productive  member  and  must  expend  time  and  energy  training  a  replacement.  But  turnover  can 
also  be  helpful,  when  a  team  loses  an  unproductive  member  and  gains  a  replacement  with 
valuable  knowledge.  Research  clarifying  the  conditions  under  which  turnover  hinders  versus 
helps  team  performance  can  thus  make  an  important  contribution  to  understanding  and 
improving  organizational  effectiveness. 

Our  simulation  work  has  other  potential  applications  as  well.  For  example,  the  new 
agentized  version  of  CONSTRUCT  can  be  used  to  examine  simultaneous  strategies  for 
stabilizing  one’s  own  group/organization  and  destabilizing  an  opposing  group/organization.  For 
example,  it  can  be  used  to  look  at  the  impact  of  (a)  recruiting  new  members  to  one’s  group  or 
increasing  the  commitment  of  current  members  and  (b)  isolating  members  of  an  opposing  group 
(e.g.,  enticing  them  to  defect,  reducing  their  operational  capability,  or  physically  eliminating 
them;  cf.  Levine,  Moreland,  &  Ryan,  1998).  As  such,  this  simulation  is  potentially  useful  for 
counter-terrorism  studies. 
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